Both Apache Kafka and Storm complement each other for accurate real-time streaming analysis of the big data in very little time. Kafka-Storm integration allows developers to easily publish streams of data using Storm topologies and run continuously.
Three main classes that oversee the integration of Kafka-Storm API are -
The BrokerHosts interface consists of two implementations - ZkHosts and StaticHosts. ZkHosts offers a fast way for tracking and maintaining the details of Kafka brokers 9n ZooKeeper while StaticHosts manually select the Kafka brokers.
The signature of ZkHosts is -
public ZkHosts(String brokerZkStr, String brokerZkPath)
public ZkHosts(String brokerZkStr)
where brokerZkStr refers to the ZooKeeper host and brokerZkPath is the ZooKeeper path for maintaining the Kafka broker and its details.
This API defines the configuration settings of the Kafka cluster and its signature is -
public KafkaConfig(BrokerHosts hosts, string topic)
where hosts are the BrokerHosts (ZkHosts / StaticHosts) and the topic corresponds to the name of the topic.
This API is an extension of KafkaConfigAPI and is responsible for storing additional ZooKeeper information. Its signature is -
public SpoutConfig(BrokerHosts hosts, string topic, string zkRoot, string id)
where zkRoot is the ZooKeeper root path and id corresponds to the state of the offsets that identifies the spout.