1.  Very easy to scale
2.  Provides high speed in both directions (both for the producers and for consumers)
3. Supports grouping of subscribers
4. Works as a cluster consisting of one or several servers.

A system that implements a distributed commit log that has the following features:

The principle of operation of the Kafka is as follows:

1. Producer writes a message to the topic partition
2. From where this message can be received by the consumer 

TOPIC is a named category in which similar messages are published.
corresponding to a specific task, distributed by partitions.

PARTITION is a numbered, physically divided part of a topic. Within
message partitions have a strict unbreakable sequence.

One topic can have as many partitions as desired. Every message within
one partition has a unique identifier (offset in terms of kafka).

PRODUCER is an entity that records data in the kafka topic.

CONSUMER - an application that reads data from the corresponding
partition topics. At the same time it is possible to read immediately from several partitions with one consumer

BROKER - A broker is an Apache Kafka server that serves requests for
add messages and read them.

OFFSET - the sequence number of the message within the partition

Brokers can be clustered to ensure resiliency and load balancing.

Apache Zookeeper is a distributed, open-source configuration, synchronization service along with naming registry for distributed applications.

CONSUMER GROUP - implementation of the publish-subscribe model, when each message being broadcasted only for one consumer from the group.
(i.e. inside the topic, each partition can only be related to one consumer group)

Why Kafka !?

• large throughput
• built-in partitions
• replication and resiliency

Good solution for applications requiring processing of large amounts of data

• Aggregation of statistical data
• Log aggregation
• Website activity tracking
• Event sourcing
• Commit-log

Hands-on​

Configuring PHP

Download kafka server http://apachemirror.

rbc.ru/pub/apache/kafka/0.10.0.0/kafka_2.11-0.10.0.0.tgz

Run zookeper & kafka

bin/zookeeper-server-start.sh config/zookeeper.properties

bin/kafka-server-start.sh config/server.properties

Let's​ do something!

bin/kafka-topics.sh --create zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic myfirsttopic

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic myfirsttopic

This is a message

This is another message

Let's​ read it!

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic myfirsttopic --from-beginning

This is a message

This is another message

Let's use PHP

Configuring

Producing

Consuming

Utility to manage and use Kafka server

• Browseclusters, brokers, topics and consumers
• View the content of messages
• Add new messages
• View consumer shifts
• Messages in pretty print
•  Sav messages from Kafka

Kafka Tool

Thank you!

My name is Ruslan.

No, I didn't read Kafka, but my consumers do!

https://kafka.apache.org/ official site

https://www.confluent.io

Kafka developers

http://www.kafkatool.com/

Kafka Tool

https://zookeeper.apache.org

ZooKeeper