Apache Kafka Powered by GlobalSolutions

Apache Kafka is primarily used to build real-time streaming data pipelines. It is an open-source system developed by the Apache Software Foundation and written in Java and Scala. Kafka runs on a cluster of one or more servers (called brokers), and the partitions of all topics are distributed across the cluster nodes. Partitions are also replicated to multiple brokers, making a Kafka cluster highly scalable, fault-tolerant, and capable of much higher throughput compared to other message brokers such as ActiveMQ and RabbitMQ.

Kafka combines three key capabilities for end-to-end event streaming:

Publish (write) and subscribe to (read) streams of events, including continuous import/export of data from other systems.
Store streams of events durably and reliably for as long as needed.
Process streams of events as they occur or retrospectively.

Why Subscribe to Our Offering in AWS Marketplace

Subscribe to our Kafka offering and spend no time on setup and configuration of the Kafka service.
Kafka is pre-installed and ready to use immediately after launching your instance.

Accessing Your AMI from AWS Marketplace

To get started with your Apache Kafka stack:

Subscribe: Purchase the Apache Kafka AMI from the AWS Marketplace.
Connect via SSH:
- SSH into the instance using the following command:
```
ssh -i your-pemfile.pem ec2-user@<public-ip-of-your-server>
```
- Once logged in you will land in the home directory.

For more information, refer to the AWS Instance Connection Guide.

Installation

Kafka is pre-installed in /opt/kafka and runs as a system service, starting automatically on boot:

Category	Status
Kafka	Running as a service (installed at `/opt/kafka`)

Testing Kafka — Quick Smoke Test

To verify that Kafka is running correctly, you can use the built-in producer and consumer scripts located in /opt/kafka/bin. We have pre-created a demo-topic for you to test with right away.

Open two separate terminal sessions (both SSH'd into your instance) — one for the producer and one for the consumer:

Terminal 1 — Start the Producer

Run the following command to start a producer on demo-topic:

/opt/kafka/bin/kafka-console-producer.sh --topic demo-topic --bootstrap-server localhost:9092

Once the producer is running, type any message and press Enter to send it.

Terminal 2 — Start the Consumer

In your second terminal, run the following command to start a consumer reading from demo-topic:

/opt/kafka/bin/kafka-console-consumer.sh --topic demo-topic --from-beginning --bootstrap-server localhost:9092

As you type messages in the producer terminal, you will see them appear in real time in the consumer terminal. This confirms that Kafka is up and working correctly.

Getting Started — Working with Topics and Messages

Creating a Topic

Navigate to the Kafka bin directory:
```
cd /opt/kafka/bin
```

Create a new topic (replace globalsolution with your own topic name):

./kafka-topics.sh --create --topic globalsolution --bootstrap-server localhost:9092

Note: Use a unique topic name when running the above command. The name globalsolution is used here as an example only.

Producing and Consuming Messages

Navigate to the Kafka bin directory:
```
cd /opt/kafka/bin
```

Start a producer and type a few messages when prompted:

./kafka-console-producer.sh --topic globalsolution --bootstrap-server localhost:9092

In a separate session, start a consumer to read messages from the topic:

./kafka-console-consumer.sh --topic globalsolution --from-beginning --bootstrap-server localhost:9092

If you need more advanced integration with alerting or other services, please reach out to us at support@theglobalsolutions.net.

AWS Cost Optimizer — CloudInsider

Our other popular offering is the AWS Cost Optimizer aka CloudInsider, available in AWS Marketplace. This service has helped our customers save significantly on AWS and other cloud spending. It is easy to subscribe and you can see the savings in minutes.

▶ Watch Demo Video Subscribe on AWS Marketplace

Support

For any questions or assistance with our AWS Marketplace offering, reach out to us at support@theglobalsolutions.net.