Apache Kafka Powered by GlobalSolutions

Apache Kafka is primarily used to build real-time streaming data pipelines. It is an open-source system developed by the Apache Software Foundation and written in Java and Scala. Kafka runs on a cluster of one or more servers (called brokers), and the partitions of all topics are distributed across the cluster nodes. Partitions are also replicated to multiple brokers, making a Kafka cluster highly scalable, fault-tolerant, and capable of much higher throughput compared to other message brokers such as ActiveMQ and RabbitMQ.

Kafka combines three key capabilities for end-to-end event streaming:

Publish (write) and subscribe to (read) streams of events, including continuous import/export of data from other systems.
Store streams of events durably and reliably for as long as needed.
Process streams of events as they occur or retrospectively.

Why Subscribe to Our Offering in AWS Marketplace

Subscribe to our Kafka offering and spend no time on setup and configuration of the Kafka service.
We are the only vendor offering built-in monitoring for the Kafka service as part of this package.
Log into Grafana and start monitoring your Kafka service immediately.

Accessing Your AMI from AWS Marketplace

To get started with your Apache Kafka stack:

Subscribe: Purchase the Apache Kafka AMI from the AWS Marketplace.
Connect via SSH:
- SSH into the instance using the following command:
```
ssh -i your-pemfile.pem ec2-user@<public-ip-of-your-server>
```
- Once logged in you will land in the home directory.

For more information, refer to the AWS Instance Connection Guide.

Installation

Both Kafka and Zookeeper are pre-installed and run as system services, starting automatically on boot:

Category	Status
Kafka	Running as a service
Zookeeper	Running as a service

Monitoring Kafka with Grafana

Retrieving Your Grafana Credentials

To get your Grafana username and password, follow these steps after SSH-ing into the instance:

Navigate to the home directory: cd /home/ec2-user
Change into the gs folder:
```
cd gs
```
Run the following command and provide your EC2 Instance ID when prompted:
```
python3 gspass.py
```
The script will display your Grafana username and password.

Accessing the Grafana Dashboard

Once you have your credentials, open a browser and navigate to:

http://<ec2-instance-public-ip>:3000

Log in with the username and password retrieved above. You will be able to see all Kafka metrics on the pre-configured dashboard.

Getting Started — Working with Topics and Messages

Creating a Topic

Navigate to the Kafka bin directory:
```
cd /usr/local/kafka/bin
```

Create a new topic (replace globalsolution with your own topic name):

./kafka-topics.sh --create --topic globalsolution --bootstrap-server localhost:9092

Note: Use a unique topic name when running the above command. The name globalsolution is used here as an example only.

Producing and Consuming Messages

Navigate to the Kafka bin directory:
```
cd /usr/local/kafka/bin
```

Start a producer and type a few messages when prompted:

./kafka-console-producer.sh --topic globalsolution --bootstrap-server localhost:9092

In a separate session, start a consumer to read messages from the topic:

./kafka-console-consumer.sh --topic globalsolution --from-beginning --bootstrap-server localhost:9092

If you need more advanced integration with alerting or other services, please reach out to us at support@theglobalsolutions.net.

AWS Cost Optimizer — CloudInsider

Our other popular offering is the AWS Cost Optimizer aka CloudInsider, available in AWS Marketplace. This service has helped our customers save significantly on AWS and other cloud spending. It is easy to subscribe and you can see the savings in minutes.

▶ Watch Demo Video Subscribe on AWS Marketplace

Support

For any questions or assistance with our AWS Marketplace offering, reach out to us at support@theglobalsolutions.net.