Kafka

*************** KAFKA ---hadoop softswares---CDK (cloud era distributed kafka) kafka4.0.1 (check in google cdk 4.1_package) compatibility with  CDH--check CDH version (support--about)

download latrest kafka from --http:/archive/cloudera.com/kafka/parcels/2.0/ (before install or upgarde always check compatibilty metrics)

CDH UI--top right --suitcase(parcel box)eaither u need to download pkgs or UI insatll

kafka--ingest streaming data into hadoop clsuter

mainly for ingest logs

works on pub0sub mechanish

3 main componets in kafka

producer--broker--consumer

versions---apace kafka (latest 2.5)--cloudera kafka (latest 4.x having packge of 2.2 of apache kafka) (CDK--confluent kafka--diff compnay only work on kafka)

upgarde cdh 5.x to 6.x

now CDP 7.0 because cdh+cdh==cdp

Install kafka packges (parcels) first from CDH UI

install -brojers in all nodes exeptp master

zookerper---plays major role in kafka

if service installtion failed---slect role log and click on full log(mostle memory issue)

Error is  : there is no meta.config file 

if you want to get any service logs ---goto cdh UI---Diagotocs --logs --error logs

version of kafka--4.1 we are using 5.12 cdh (downgrade kafka version)

steps to follow install kafka

1.check the compatibioty of cdh version ]

2.download kafka paks -distribute and activate parcels

3.click on add servicwe--select kafaka

4.provide server details to configure kafka brokers

5. add gateways in edge nodes

7.click on continue to start the kafaka service

8.If kafka brokers are failed to start then login to cdh --click on --kafka--configuratons--serch for heap memory--change the memory for kafka brokers to 1Gb--restrt kaftka service

**** kafka theory ******