Kafka

Producer:

1. Fire & forget, Sync, Async with callback(Future, future.get())
2. 2 kind of errors : retriable(connection error, no leader) & non-retriable(message too large)
3. RecordMetadata rec = producer.send(new ProducerRecord()); rec has offset of message
4. ProducerRecord needs topic name, key serializer, value serializer, bootstrap.servers(at least 2 so that if one is down other can take over, rest will be discovered by them)
5. 
   

Consumers:

1. One consumer group per application. If #consumers > #partitions, rest of the consumers will remain idle.
2. One consumer per thread is the rule.
3. How does consumer commit offset:
1. auto commit after every 5 seconds during poll.
2. or commit manually - consumer.commitSync()
3. or commitAsync() with callback
4. commitAsync() for regular commits, commitSync() in finally block
5. commitSync and commitAsync can be called with explicit topic,partition,offset too.
4. RebalanceListener
1. OnPartitionsRevoked
2. OnPartitionsAssigned
5. SeekBeginning, SeekEnd, seek specific offset
6. Consumer clean exit: consumer.wakeup() from shutdownhook. consumer.close(). wakeupException().
7. Single Consumer also possible as opposed to the Consumer Group.
8. 
   

Serialization:

1. Custom serializers
2. Avro: Schema can be changed without affecting existing messages

Partitioning

1. Default: hash
2. Custom
3.