Which partition id to use when subscribe a topic.

[wenx1]

Hi guys, I am following the example to create a simple C++ Consumer.

RdKafka::Consumer::create

And then using its

RdKafka::ErrorCode resp = ptrConsumer->start(topic, RdKafka::Topic::PARTITION_UA, OFFSET_BEGINNING);

But it returns an error RD_KAFKA_RESP_ERR__UNKNOWN_PARTITION because RdKafka::Topic::PARTITION_UA is less than 0.

Do I need to first know which partition id to use before subscribing a topic? I don't see it is required when I wrote a same consumer with python code. However I got this error when using C++..

And in the code of librdkafka, I did see the code will check the partition id.

static RD_UNUSED
int rd_kafka_consume_start0 (rd_kafka_itopic_t *rkt, int32_t partition,
				    int64_t offset, rd_kafka_q_t *rkq) {
	shptr_rd_kafka_toppar_t *s_rktp;

	if (partition < 0) {
		rd_kafka_set_last_error(RD_KAFKA_RESP_ERR__UNKNOWN_PARTITION,   <------------------------------------
					ESRCH);
		return -1;
	}

        if (!rd_kafka_simple_consumer_add(rkt->rkt_rk)) {
		rd_kafka_set_last_error(RD_KAFKA_RESP_ERR__INVALID_ARG, EINVAL);
                return -1;
        }

[rolandyoung]

The RdKafka::Consumer class provides a rather specialised, low-level API. You should use the RdKafka::KafkaConsumer class instead.
Despite the name, you probably want to use rdkafka_complex_consumer_example.cpp rather than rdkafka_example.cpp as a starting point.

[paul-bormans-pcgw]

THANK YOU !!!
I was seeing all kinds of wierdness, like my consumer not popping up in kafdrop and consumer_lag always being -1... Switching to the high level consumer solved all of this.