Recent questions tagged kafka

0 votes

1.4k views

1 answer

kafka consumer api - Apache Beam, KafkaIO at least once semantics

We are implementing a pilot that reads from Kafka and writes to BigQuery. Simple pipeline: KafkaIO.read ... /questions/65951040/apache-beam-kafkaio-at-least-once-semantics...

asked Oct 7, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.4k views

1 answer

kafka consumer api - How child json attributes to parent json in nifi

I have json nifi flow content like below { "name": "Rohit", "stock_details": "{'GOOG': '234', 'AMZN': ... .com/questions/66061807/how-child-json-attributes-to-parent-json-in-nifi...

asked Oct 6, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.2k views

1 answer

kafka message key as key field/column in HDFS

So I use the debezium key.field.name in my MySQL source connector to add a field into my topic. The ... .class": "io.confluent.connect.hdfs.partitioner.TimeBasedPartitioner" } }...

asked Feb 19, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.3k views

1 answer

kafka 一个分区partition 只能有一个消费者吗？

请问一个topic可以对应一个消费组，一个消费组可以有多个消费者，消息发布的时候，只发送给消费组中的某一个消费者，如果topic只有一个partition的话，消费组里面可以有多个消费者吗？还是只能有一个消费者？...

asked Feb 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.5k views

1 answer

kafka SASL_PLAIN SCRAM is fail in spring boot consumer

I try kafka authentication SASL_PLAINTEXT / SCRAM but Authentication failed in spring boot. i try change ... factory.setConsumerFactory(consumerFactory()); return factory; }...

asked Feb 6, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

kafka 的问题

想问下这个logs 是存储什么信息的，文件过大的时候怎么可以清理下？...

asked Feb 5, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.4k views

1 answer

kafka acks和retries是否会相互作用？

kafka acks=0的时候，producer是不管写入数据是否成功的。那如果这时候我们再去设置retries还有意义么？类似如果acks=0，retries=3，消息发送给broker过程中broker宕机了，这时候是否会自动retry？...

asked Jan 27, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

2.4k views

1 answer

kafka java process consuming way large memory specially thread called org.springframework.kafka.KafkaListenerEndpointContainer#1-0-C-1

org.springframework.kafka.KafkaListenerEndpointContainer#1-0-C-1 consume large memory how to reduce it I'm using ... ))) userService.applicationExpire(kafkaEvent.getEventData()); }...

asked Jan 27, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags