How to Install Kafka on Ubuntu Server Latest

Apache Kafka is a popular open-source messaging system that streams events in real-time. In this tutorial, we will walk through how to install Kafka from the official Apache Kafka website on an Ubuntu server.

Prerequisites

Before installing Kafka on Ubuntu, there are a few prerequisites you should have installed:

Java: Kafka requires Java 8 or higher to run.
ZooKeeper: Kafka uses ZooKeeper to manage its cluster. You can install ZooKeeper by following these instructions.

Step 1: Download Kafka

First, go to the Apache Kafka website and download the latest stable release of Kafka. At the time of writing, the latest version is Kafka 2.7.1.

wget https://downloads.apache.org/kafka/2.7.1/kafka_2.13-2.7.1.tgz

When the download is complete, extract the contents of the file to the /opt directory:

sudo tar -xzf kafka_2.13-2.7.1.tgz -C /opt

This will create a directory called kafka_2.13-2.7.1 in the /opt directory.

Step 2: Start Kafka

Before starting Kafka, ensure that ZooKeeper is running. If you installed ZooKeeper on the same server, run:

sudo systemctl start zookeeper

To start Kafka, open a new terminal window and navigate to the Kafka directory:

cd /opt/kafka_2.13-2.7.1

Next, run the following command:

sudo ./bin/kafka-server-start.sh config/server.properties

This will start Kafka with the default configuration. If you want to customize Kafka's configuration, edit the config/server.properties file before starting Kafka.

Step 3: Create a Kafka Topic

Once Kafka is running, you can create a topic to start publishing messages. To create a new topic, run:

sudo ./bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic my-topic

This will create a topic called my-topic with one partition and one replication factor.

Step 4: Publish and Consume Messages

To publish messages to a topic, run:

sudo ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic my-topic

This will open a console where you can enter messages to publish to the topic.

To consume messages from a topic, open a new terminal window and run:

sudo ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my-topic --from-beginning

This will open a console where you can see the messages published to the my-topic topic.

That's it! You now have Apache Kafka running on your Ubuntu server, and you can start building your own real-time data streaming applications.