The Status Airflow plugin makes it easy to communicate confidence about your data system to manager, executives and other stakeholders in your organization. It improves trust in underlying data by increasing transparency.

For a good walkthrough of the tool, head over to our Medium article about the rb_status_plugin

These are instructions for importing this plugin into an existing airflow workspace.
To start, navigate to the root of your airflow workspace.
If you don't have an existing workspace, you can download the sample:

>git clone https://github.com/Raybeam/rb_test_airflow/ sample_workspace
>cd sample_workspace

The deployment environments are:
Local Deploy
Astronomer Deploy
Google Cloud Composer Deploy

Clone a sample airflow workspace (if you dont have an existing airflow repository).

git clone https://github.com/Raybeam/rb_test_airflow/ sample_workspace
cd sample_workspace

Clone deploy script into local workspace

git clone https://github.com/Raybeam/rb_plugin_deploy plugins/rb_plugin_deploy

Run deploy script.

./plugins/rb_plugin_deploy/deploy.sh

> python -m venv .

By putting the AIRFLOW_HOME env in the bin/activate file, you set the path each time you set up your venv.

> echo "export AIRFLOW_HOME=$PWD" >> bin/activate

> source bin/activate

> git clone https://github.com/Raybeam/rb_status_plugin plugins/rb_status_plugin

> cat plugins/rb_status_plugin/requirements.txt >> requirements.txt
> pip install -r requirements.txt

> pip install apache-airflow

> airflow initdb

> airflow create_user -r Admin -u admin -e [email protected] -f admin -l user -p admin

Move over the main rb status DAG and sample DAGs (if wanted)

> plugins/rb_status_plugin/bin/setup init

> plugins/rb_status_plugin/bin/setup add_samples

Only the DAG works from the rb status plugin binary right now.

> plugins/rb_status_plugin/bin/setup add_samples --dag_only

In the root directory of your airflow workspace, open airflow.cfg and set rbac=True.

>airflow webserver

In a new terminal, navigate to the same directory.
>source bin/activate
>airflow scheduler

In a web brower, visit localhost:8080.
If you see a tab for "Status" in the header, then the installation was a success.

Follow the local deploy instructions for configuring your local environment.

Either Control+C or closing the terminal's window/tab should work to turn either of them off.

Download astronomer package following their tutorial.

In your working directory > astro dev init

> astro dev start

In a web brower, visit localhost:8080.
If you see a tab for "Status" in the header, then the installation was a success.

> git clone https://github.com/Raybeam/rb_status_plugin plugins/rb_status_plugin

Install the gcloud SDK and configure it to your Cloud Composer Environment.

>gcloud auth login

>gcloud config set project <your Google Cloud project name>

>gcloud composer environments update ENVIRONMENT_NAME --location LOCATION --update-pypi-packages-from-file=plugins/rb_status_plugin/requirements.txt

ENVIRONMENT_NAME is the name of the environment.
LOCATION is the Compute Engine region where the environment is located.
It may take a few minutes for cloud composer to finish updating after running this command.

>gcloud composer environments update ENVIRONMENT_NAME --location LOCATION --update-airflow-configs \  
	webserver-rbac=False,\  
	core-store_serialized_dags=False,\  
	webserver-async_dagbag_loader=True,\  
	webserver-collect_dags_interval=10,\  
	webserver-dagbag_sync_interval=10,\  
	webserver-worker_refresh_interval=3600

ENVIRONMENT_NAME is the name of the environment.
LOCATION is the Compute Engine region where the environment is located.

Add rb_status dag to dags folder:

 >gcloud composer environments storage dags import\  
    --environment ENVIRONMENT_NAME \
    --location LOCATION \
    --source SOURCE/setup/rb_status.py

Add rb_status plugin to plugins folder:

>gcloud composer environments storage plugins import\
    --environment ENVIRONMENT_NAME \
    --location LOCATION \
    --source SOURCE

ENVIRONMENT_NAME is the name of the environment.
LOCATION is the Compute Engine region where the environment is located.
SOURCE is the absolute path to the local directory (full-path/plugins/rb_status_plugin/).