Resetting an Airy Kafka streaming app
How to reset an Airy component.
The Airy Core components are Kafka streaming apps which consume and produce to Kafka. Each of them has its own consumer group that can be reset, to re-read all the events and messages from the beginning or from a certain offset.
Consumer group information
Information about the Kafka consumer groups used by Airy Core can be printed out with the following command:
kubectl exec -it kafka-0 -- kafka-consumer-groups.sh --bootstrap-server kafka-headless:9092 --list
To retrieve information about the particular consumer group run:
kubectl exec -it kafka-0 -- kafka-consumer-groups.sh --bootstrap-server kafka-headless:9092 --describe --group $CONSUMER_GROUP | grep -v TOPIC | grep -v "Consumer group" | awk '{ print $2; }' | sort | uniq | grep -v repartition | grep -v KTABLE | tr "\n" "," | sed 's/,$/\n/g' | sed 's/^,//g'
Reset the Kafka streaming app
Once you have the information about the consumer group, you can proceed with reset the kafka-streaming app.
First scale down the particular component:
kubectl scale deployment $DEPLOYMENT --replicas=0
Then reset the Kafka streaming app:
kubectl exec -it kafka-0 -- kafka-streams-application-reset.sh --bootstrap-servers kafka-headless:9092 --application-id $CONSUMER_GROUPS --to-earliest --input-topics $INPUT_TOPICS
Then scale the app back up:
kubectl scale deployment $DEPLOYMENT --replicas=1
Example
For example, to reset the api-communication streaming app, run the following commands:
export DEPLOYMENT="api-communication"
kubectl scale deployment $DEPLOYMENT --replicas=0
export CONSUMER_GROUP="api.CommunicationStores"
INPUT_TOPICS=$(kubectl exec -it kafka-0 -- kafka-consumer-groups.sh --bootstrap-server kafka-headless:9092 --describe --group $CONSUMER_GROUP | grep -v TOPIC | grep -v "Consumer group" | awk '{ print $2; }' | sort | uniq | grep -v repartition | grep -v KTABLE | tr "\n" "," | sed 's/,$/\n/g' | sed 's/^,//g')
kubectl exec -it kafka-0 -- kafka-streams-application-reset.sh --bootstrap-servers kafka-headless:9092 --application-id $CONSUMER_GROUP --to-earliest --input-topics $INPUT_TOPICS
kubectl scale deployment $DEPLOYMENT --replicas=1