Apache Airflow Powered by GlobalSolutions

Apache Airflow is an open-source tool to programmatically author, schedule, and monitor workflows. It is one of the most robust platforms used by Data Engineers for orchestrating workflows or pipelines. With Airflow, users can author workflows as Directed Acyclic Graphs (DAGs) of tasks. Airflow's rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed.

Airflow connects with multiple data sources and can send alerts via email or Slack when a task completes or fails. It is distributed, scalable, and flexible, making it well-suited to handle the orchestration of complex business logic.

Airflow is a distributed system consisting of the following components:

Webserver — Provides a user interface for managing Airflow workflows.
Scheduler — Responsible for scheduling and running Airflow tasks.
Worker — Worker nodes that execute Airflow tasks.
Executor — Responsible for running tasks on worker nodes.
Metadata database — Stores information about Airflow workflows, tasks, and other entities.

Why Subscribe to Our Offering in AWS Marketplace

We update the software constantly to the latest version to address security issues.
Customers can kick-start their core work right away with our pre-packaged AMIs.
Production-ready application stacks.

Accessing Your AMI from AWS Marketplace

To get started with your Apache Airflow stack:

Subscribe: Purchase the Apache Airflow AMI from the AWS Marketplace.
Connect via SSH:
- SSH into the instance using the following command:
```
ssh -i yourpemfile.pem ec2-user@<public-ip-of-your-server>
```
- Once logged in you will land in the home directory.

For more information, refer to the AWS Instance Connection Guide.

Installation Paths and Versions

Apache Airflow is installed in the following location:

Category	Packages	Version	Location
Apache Airflow	Server	2.9.1	`/home/ec2-user/airflow`

Connecting to the Application

To access the Apache Airflow web interface from your local machine, open a browser and navigate to:

http://<server_ip>:8080

Username	Password
admin	global

Important: Change the default password immediately after your first login to secure your Airflow instance.

Getting Started with DAGs

Directed Acyclic Graphs (DAGs) define your workflows in Airflow. Follow these steps to get started:

Access the Airflow web interface using the URL and credentials above.
Create DAGs to define your workflows.
Monitor task execution and view task logs from the UI.
Trigger and pause workflows as needed.

Sample DAG

A sample DAG is pre-installed at /home/ec2-user/airflow/dag/my_first_dag.py. To view it running, go to the Airflow console and search for sample_dag — you should see it in a running state.

Note: If you need help creating new DAGs, the GlobalSolutions team can assist — however this will be a separate engagement.

AWS Cost Optimizer — CloudInsider

Our other popular offering is the AWS Cost Optimizer aka CloudInsider, available in AWS Marketplace. This service has helped our customers save significantly on AWS and other cloud spending. It is easy to subscribe and you can see the savings in minutes.

▶ Watch Demo Video Subscribe on AWS Marketplace

Support

For any questions or assistance with our AWS Marketplace offering, reach out to us at support@theglobalsolutions.net.