Apache Kafka Problems with offset

Question

I'm using confluent kafka library to create a distributed system, but I'm failing to understand some principles of Kafka itself. Lets say right now I'm working with a Central, that has to listen to different customers. What i wanna do is that when the program Central is shut off, whatever customer program sends, it won't be received by the Central when I execute this program again. The way I configure it was the following:

PRODUCTOR CUSTOMER TO CENTRAL

producer_conf = {
'bootstrap.servers': f'{broker_ip}:9092',
'retries': 5,
'retry.backoff.ms': 1000
}
self.producer = Producer(producer_conf)
self.requests = self.load_requests(self.request_file)

CONSUMER CENTRAL LISTENING TO CUSTOMER:

consumer_conf = {
'bootstrap.servers': f'{broker_ip}:9092',
'group.id': 'central_group',
'auto.offset.reset': 'latest',
'enable.auto.commit': True,
}
self.consumer = Consumer(consumer_conf)
self.consumer.subscribe(['taxirequests'])

CREATION OF THE TÓPICO (manually): bin/kafka-topics.sh --create --topic taxirequests --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

Could someone help me? Thanks a lot!

Your problem isn’t clear to me. What is central, what’s customer, what’s shut down and what’s listening? Please use standard Kafka jargon (producer, consumer, topic etc.) to state your problem. — Abhijit Sarkar
– Abhijit Sarkar, Commented Dec 12, 2024 at 23:16

Silpa · Accepted Answer · 2024-12-12 22:54:00Z

0

Try to add auto.offset.reset=latest and check how it works.

answered Dec 12, 2024 at 22:54

Silpa

163 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Apache Kafka Problems with offset

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related