July 24, 2017

how to start the Apache spark with Kafka application in windows?

To configure Apache Kafka in Windows:
https://www.youtube.com/watch?v=MVdGjs0YQBQ
https://www.youtube.com/watch?v=LS6WiHs_m5w

Apache Spark With Kafka:
https://spark.apache.org/docs/latest/streaming-kafka-integration.html
http://kafka.apache.org/documentation.html

Integrating Spark Streaming with Apache Kafka:
https://www.youtube.com/watch?v=5V-Wtfrc38U&t=621s

Advanced Apache Spark Training - Sameer Farooqui (Databricks):
https://www.youtube.com/watch?v=7ooZ4S7Ay6Y

Intro to Apache Spark Streaming | NewCircle Training:
https://www.youtube.com/watch?v=2STfulBcorA

How to start the zookeeper/kafka application after the installation with path configuration:

Zookeeper server start configuration:
C:\Programs\zookeeper-3.4.10\bin>zkserver

kafka server :
C:\Programs\kafka_2.12-0.11.0.0>.\bin\windows\kafka-server-start.bat .\config\server.properties

kafka-topics:
C:\Programs\kafka_2.12-0.11.0.0\bin\windows>kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test7

or

C:\Programs\kafka_2.12-0.11.0.0\bin\windows>kafka-topics.sh --describe --zookeeper localhost:2181 --topic test

kafka-console-producer:
C:\Programs\kafka_2.12-0.11.0.0\bin\windows>kafka-console-producer.bat --broker-list localhost:9092 --topic test7
> my test message 7


kafka-console-consumer:
C:\Programs\kafka_2.12-0.11.0.0\bin\windows>kafka-console-consumer.bat --zookeeper localhost:2181 --topic test7
> my test message 7

No comments:

Post a Comment

I'm certainly not an expert, but I'll try my hardest to explain what I do know and research what I don't know.

My Favorite Site's List

#update below script more than 500 posts