Two Kafka Tips

TO change the heap options in KAFKA Set on the commandline or edit kafka-env.sh on CDP or IOP Add KAFKA_HEAP_OPTS=”-Xmx64G -XMs64G” From IBM, Running with IBM JVM To resolve this issue: Edit the <kafka_home>/bin-kafka-run-class.sh file. Locate the following line: KAFKA_GC_LOG_OPTS=”-Xloggc:$LOG_DIR/$GC_LOG_FILE_NAME -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps” To enable verbose logging, change it to: KAFKA_GC_LOG_OPTS=”-Xverbosegclog:$LOG_DIR/$GC_LOG_FILE_NAME -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps […]

Kafka Kerberos Debug

I’ve spent a lot of time with Kerberos recently. If you are debugging kerberos and kafka, try this before starting Kafka add -Dsun.security.krb5.debug=true to the KAFKA_HEAP_OPTS. export KAFKA_HEAP_OPTS=”-Xmx1G -Xms1G -Djava.security.auth.login.config=/opt/kafka/security/kafka_server_jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf -Dlog4j.properties=file:///opt/kafka/config/log4j.properties -Dsun.security.krb5.debug=true” /opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties You get a rich log file   root@broker:/# cat zookeeper.log [2017-05-20 20:25:33,482] INFO Reading configuration from: /opt/kafka/config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) [2017-05-20 20:25:33,495] […]