- Fire & forget, Sync, Async with callback(Future, future.get())
- 2 kind of errors : retriable(connection error, no leader) & non-retriable(message too large)
- RecordMetadata rec = producer.send(new ProducerRecord()); rec has offset of message
- ProducerRecord needs topic name, key serializer, value serializer, bootstrap.servers(at least 2 so that if one is down other can take over, rest will be discovered by them)
- One consumer group per application. If #consumers > #partitions, rest of the consumers will remain idle.
- One consumer per thread is the rule.
- How does consumer commit offset:
- auto commit after every 5 seconds during poll.
- or commit manually - consumer.commitSync()
- or commitAsync() with callback
- commitAsync() for regular commits, commitSync() in finally block
- commitSync and commitAsync can be called with explicit topic,partition,offset too.
- SeekBeginning, SeekEnd, seek specific offset
- Consumer clean exit: consumer.wakeup() from shutdownhook. consumer.close(). wakeupException().
- Single Consumer also possible as opposed to the Consumer Group.
- Custom serializers
- Avro: Schema can be changed without affecting existing messages
- Default: hash