Skip to content

Commit 8dee951

Browse files
authored
Update README.md
1 parent d17dd12 commit 8dee951

File tree

1 file changed

+91
-1
lines changed

1 file changed

+91
-1
lines changed

README.md

Lines changed: 91 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,92 @@
11
# kafka-nodejs-tutorial
2-
A sample application using Kafka, Node.js and Docker
2+
# A sample application using Kafka, Node.js and Docker
3+
4+
## 1. About Kafka Streaming
5+
6+
Below is the discription from the official Kafka web site.
7+
8+
Apache Kafka® is a distributed streaming platform.
9+
10+
It publishes and subscribes to streams of records, similar to a message queue or enterprise messaging syste
11+
12+
Kafka is generally used for two broad classes of applications:
13+
14+
Building real-time streaming data pipelines that reliably get data between systems or applications
15+
Building real-time streaming applications that transform or react to the streams of data
16+
17+
To learn more about Kafka, please visit [Kafka official web site](https://kafka.apache.org/)
18+
19+
Kafka is build in Java, it does provide APIs from many other lanhuages.
20+
21+
Since I am a Javascript developer, I decided to use Node.js for this sample application.
22+
23+
There are a few libraries for Node.js, some popular libraries include:
24+
25+
* [node-rdkafka](https://github.com/Blizzard/node-rdkafka)
26+
27+
* [Kafka-node](https://www.npmjs.com/package/kafka-node)
28+
29+
* [KafkaJS](https://www.npmjs.com/package/kafkajs)
30+
31+
I use KafkaJs in this example. The main reason is it has better documentation then the other two.
32+
33+
34+
35+
## 2. Use case - track travellers who may infected by Coronarirus virus
36+
37+
We are going to build an applicaiton, to track petential travellers who may have been infected by Coronavirus.
38+
39+
1. We check travellers temperature at major airports.
40+
2. Temperature and traveller's personal infomration are sent to the Kafka queue (as producer). Note every airport is a prodcuer.
41+
3. We have another app check every message sent to the queue (near real time), and alert is triggered if we a passenger has higher temperature and has overseas travel history.
42+
43+
44+
45+
## 3. Build local Kafka environemnt using Docker and Container
46+
47+
In order to run the application locally, we need at least two docker imagers
48+
49+
* zookeeper
50+
* Kafka Broker
51+
52+
In the real envirnment, you will need multiple Kafka brokers, to form a cluster.
53+
54+
In stead of build my own docker imagers, I cloned the code from https://github.com/wurstmeister/kafka-docker, which give me everything I need to run Kafka on my local container
55+
56+
57+
58+
59+
60+
```yml
61+
version: "2"
62+
services:
63+
zookeeper:
64+
image: wurstmeister/zookeeper
65+
ports:
66+
- "2181:2181"
67+
kafka:
68+
build: .
69+
ports:
70+
- "9092:9092"
71+
environment:
72+
KAFKA_ADVERTISED_HOST_NAME: localhost
73+
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
74+
KAFKA_CREATE_TOPICS: "flight:1:1"
75+
volumes:
76+
- /var/run/docker.sock:/var/run/docker.sock
77+
78+
```
79+
80+
81+
82+
83+
84+
Reference: https://github.com/wurstmeister/kafka-docker
85+
86+
87+
## 4. Run the application
88+
89+
90+
91+
## References
92+

0 commit comments

Comments
 (0)