Air pollution monitoring system built with AWS IoT and Greengrass

by sz3jdii
0 comment 15 read
AWS IoT

IoT (eng. Internet of Things) is becoming the most important approach in creating consumer and industrial solutions. In this guide we’ll try to successfully connect the IoT device with cloud. To do that, we’ll build an air pollution monitoring system which will notify us about our air quality, all of that will be achieved thanks to NanoPi Neo V1.4, SDS011 sensor, AWS IoT, AWS Greengrass, AWS CloudWatch and Amazon SNS.

AWS offers Internet of Things (IoT) services and solutions to connect and manage billions of devices.
AWS offers Internet of Things (IoT) services and solutions to connect and manage billions of devices.

Prerequisite

Before we start you need to make sure that you have the following:

IoT project overview

Air pollution monitoring system built with AWS IoT
Air pollution monitoring system built with AWS IoT

What we’ll try to build is already mentioned air pollution monitoring system. NanoPi device will be connected by UART-USB connector with SDS011 sensor, which measures PM2.5 and PM10 markers of air pollution. NanoPi will be connected to the AWS IoT core by AWS Greengrass. Using AWS Greengrass components we’ll deploy at edge, simple Python app which will collect and publish (using AWS Greengrass MQTT component) air pollution metrics to the cloud. Using the AWS IoT Message Routing we’ll send measured data to the AWS CloudWatch, and there we’ll create a CloudWatch Alarm connected to SNS topic. Alarm will be triggered when air pollution norms will be exceeded.

The connection between device and sensor should look similar to the below image:

NanoPi Neo V1.4 + SDS011 project
NanoPi Neo V1.4 + SDS011 project

Ubuntu 20.04 installation on NanoPi

NanoPi NEO v1.4
NanoPi NEO v1.4

The first what we need to do is installing Linux software on our IoT device. I’ve created a separate post mentioning how to install Ubuntu 20.04 on NanoPi Neo V1.4 in a ‘headless’ mode. Check the link here. Next, we need to ensure that we have below things installed in order to run Greengrass on the device:

  • Python 3.8+ with pip (guide is located here)
  • cmake, curl, zip, unzip, git
  • Java 11 / AWS Correto 11 (if you find your device incompatible with Correto, use OpenJDK made by Azul)

IAM User creation

First of all we need to setup IAM user which will be used for performing Greengrass operations. Follow the link to go to the AWS IAM console.

AWS IAM User creation
AWS IAM User creation

Create a user with Access key – Programmatic access and assign him AdministratorAccess policy (WARNING! We use admin access only because of the fact that we are creating development environment and we’d like to ensure that lack of permission won’t disturb our workshops. Don’t use AdministratorAccess policy in production environment!).

Exporting AWS Access key id and secret key
Exporting AWS Access key id and secret key

Copy the Access key ID and Secret access key and save them as environment variables inside NanoPi device and local device using the following commands:

export AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<AWS_SECRET_ACCESS_KEY>

Installing AWS IoT Greengrass Development Kit Command-Line on local computer

Our Python app is a Greengrass component built to collect data from SDS011 sensor. Thanks to the Greengrass we’ll build the component on our local computer, then using the AWS IoT Greengrass Development Kit Command-Line (GDK) we’ll publish this component code to S3 bucket and after that we’ll run this component on our NanoPi device using the AWS Console only.

AWS IoT Greengrass Development Kit Command-Line
AWS IoT Greengrass Development Kit Command-Line

First of all, let’s download to our local computer my Python app which purpose is to monitor air pollution and connect using MQTT with AWS IoT:

git clone https://github.com/sz3jdii/air_pollution_iot_metter.git

Next, we need to build our artifact in order to publish it to an S3 bucket. To do this we need AWS IoT GDK. Let’s install it:

pip install -U git+https://github.com/aws-greengrass/[email protected]

If you’ll find any problems at this point, please take a look at developer’s guide here.

Type below command to check if everything is working as expected:

gdk --version

You should see similar view:

GDK version check
GDK version check

Next, let’s jump to the directory where we’ve cloned our project code and install all Python requirements.

pip install -r requirements.txt

After installing all of the requirements, we can proceed with building and publishing artifact to the S3 bucket, but first let’s take a quick look how to project looks like:

AWS Greengrass Artifact project files
AWS Greengrass Artifact project files

gdk-config.json is a file where we configure how the building of component should work. In our project, we are telling GDK that we want to have a zip file which will be published to the S3 bucket prefixed with iot-greengrass-air-pollution. NEXT_PATCH is used to automatically increase the version number of our package by GDK.

gdk-config.json:

{
  "component": {
    "com.cloudybarz.airPollutionMetter": {
      "author": "sz3jdii",
      "version": "NEXT_PATCH",
      "build": {
        "build_system": "zip"
      },
      "publish": {
        "bucket": "iot-greengrass-air-pollution",
        "region": "eu-central-1"
      }
    }
  },
  "gdk_version": "1.0.0"
}

recipe.json is a file that defines a component’s details, dependencies, artifacts and lifecycles. In the ComponentConfiguration section we define a rule to allow the component to publish to the IoT MQTT topic. In Manifests section we define steps which our component needs to do on each selected platform. The component Lifecycle specifies the commands to run, install and shut down the component. Artifacts section defines what files are needed by our component this section also defines location of them.

recipe.json:

{
  "RecipeFormatVersion": "2020-01-25",
  "ComponentName": "com.cloudybarz.airPollutionMetter",
  "ComponentVersion": "COMPONENT_VERSION",
  "ComponentDescription": "Simple air pollution metter built with AWS Greengrass.",
  "ComponentPublisher": "Sz3jdii",
  "ComponentDependencies": {
  },
  "ComponentConfiguration": {
    "DefaultConfiguration": {
      "accessControl": {
        "aws.greengrass.ipc.mqttproxy": {
          "com.cloudybarz.airPollutionMetter:mqttproxy:1": {
            "policyDescription": "Allows access to publish/subscribe to all topics.",
            "operations": [
              "aws.greengrass#PublishToIoTCore",
              "aws.greengrass#SubscribeToIoTCore"
            ],
            "resources": [
              "*"
            ]
          }
        }
      }
    }
  },
  "Manifests": [
    {
      "Platform": {
        "os": "linux"
      },
      "Lifecycle": {
        "Install": {
          "RequiresPrivilege": true,
          "script": "pip3 install -r {artifacts:decompressedPath}/air_pollution_iot_metter/requirements.txt"
        },
        "Run": {
          "RequiresPrivilege": true,
          "script": "python3 -u {artifacts:decompressedPath}/air_pollution_iot_metter/main.py"
        }
      },
      "Artifacts": [
        {
          "URI": "s3://iot-greengrass-air-pollution/air_pollution_iot_metter.zip",
          "Unarchive": "ZIP",
          "Permission": {
            "Read": "ALL"
          }
        }
      ]
    }
  ]
}

The /src directory contains the logic of the application. Now, as you know what the structure of AWS Greengrass components looks like, we can build and upload our component to an S3 bucket. First of all we need to assign a policy which allows access to an S3 bucket by the Greengrass execution role. Create a file named: component-artifact-policy.json with the below content.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::iot-greengrass-air-pollution*/*"
    }
  ]
}

Run the below command on the local machine:

aws iam create-policy \
  --policy-name MyGreengrassV2ComponentArtifactPolicy \
  --policy-document file://component-artifact-policy.json

Copy the policy Amazon Resource Name (ARN) from the policy metadata in the output. You will use this ARN to attach this policy to the NanoPi Greengrass device role in the next step. Replace the REPLACE_WITH_ARN with the ARN copied and run the below command:

aws iam attach-role-policy \
  --role-name GreengrassV2TokenExchangeRole \
  --policy-arn arn:aws:iam::REPLACE_WITH_ARN:policy/MyGreengrassV2ComponentArtifactPolicy

If the command has no output, it succeeded.

Now, let’s build and publish our component. From the directory of the project on our local machine run the below commands:

gdk component build
gdk component publish

If everything went well, you should see below output:

gdk component build && gdk component publish
gdk component build && gdk component publish

You can also check if an S3 bucket was created inside AWS console.

AWS S3 bucket with component's artifacts
AWS S3 bucket with component’s artifacts

Connecting NanoPi to the cloud using AWS Greengrass

If we have everything setup well, we can proceed with connecting our IoT setup with the cloud. Let’s jump into the AWS IoT Greengrass console here.

AWS Greengrass Console
AWS Greengrass Console

Follow the “Set up one core device” link and fill the form with our device information.

AWS Greengrass device registration
AWS Greengrass device registration

Let’s run both commands on the NanoPi device (login to it using SSH) to download and install Greengrass components.

Installing nucleus and Greengrass things
Installing nucleus and Greengrass things

If everything went well, we should see our device registered in the Greengrass core devices list.

AWS Greengrass Core Devices List
AWS Greengrass Core Devices List

Deploying the component to the IoT device

Let’s summarize what we should achieve for now. We have set up a connection between the IoT device (NanoPi + SDS011) and AWS IoT using the Greengrass. Thanks to the Greengrass GDK we’ve also created our first component which now waits on an S3 bucket for being deployed to the IoT core device. To do that, let’s jump to the Deployments section in the AWS Greengrass console here.

AWS Greengrass deployments console
AWS Greengrass deployments console

Check the box near the name of the default deployment and click Revise button.

AWS Greengrass Deployment Step 1
AWS Greengrass Deployment Step 1

Leave the Step 1 form as it is. Click the Next button to go to the next step.

AWS Greengrass Deployment Step 2
AWS Greengrass Deployment Step 2

Select the component which we’ve published using GDK from our local machine. After that, click the Next button and go straight to the Review Step (Step 5).

AWS Greengrass Deployment Step 5
AWS Greengrass Deployment Step 5

Lastly, click the Deploy button and wait a little.

Logs and monitoring

AWS Greengrass Logs and monitoring
AWS Greengrass Logs and monitoring

If you want to monitor the status of the component and device you can focus on below two files located in IoT device. com.cloudybarz.airPollutionMetter.log represents the state and outputs from the components which we’ve just deployed.

tail -f /greengrass/v2/logs/com.cloudybarz.airPollutionMetter.log 

greengrass.log file represents the state of Greengrass services running on the IoT device.

tail -f /greengrass/v2/logs/greengrass.log

Subscribing to MQTT topic

AWS Greengrass Active deployment
AWS Greengrass Active deployment

If our deployment went well, we can use builtin MQTT test client located in the AWS IoT console here. Our Python app, publishes to the #pollution topic metrics collected from the sensor. Let’s subscribe to that topic to see if we can process them later.

After a while we should see messages coming to the client which will mean that everything is working fine.

Routing the MQTT messages to the CloudWatch service

AWS Cloudwatch service
AWS Cloudwatch service

Now, as we have data coming properly to the cloud, we’d like to visualize it in the form of a graph and notify user if air pollution will be exceeded. There’s a great AWS IoT feature called Message Routing, which can allow us to redirect MQTT messages to other AWS service such as AWS CloudWatch. The great feature of Message Routing is also creating redirection rules based on an SQL query. Let’s jump to the Message routing console here. Click the Create rule button and name the rule “Publish_air_pollution_metrics_to_Cloudwatch”.

AWS Message routing rule creation step 1
AWS Message routing rule creation step 1

Click the Next button to go to the Step 2. In this step we need to set an SQL query to grab only MQTT messages with selected conditions and from selected topic. In our case we want to forward every message sent to the ‘pollution’ MQTT topic so we can use below query.

SELECT * FROM 'pollution'
AWS Message routing rule creation step 2
AWS Message routing rule creation step 2

Click the Next button and go to the Step 3 where we’ll set actions. We need to set two actions, one for each metric. Let’s see how properly fill the forms.

AWS Message routing rule creation step 3 action 1
AWS Message routing rule creation step 3 action 1

If you don’t have IAM role, create one using the Create new role button. Let’s setup 2. action similar to the first one.

AWS Message routing rule creation step 3 action 2
AWS Message routing rule creation step 3 action 2

We want also to know if something goes wrong so we can set up an error action which will store an error message in Cloudwatch Logs. In the same form as above, fill the error action fields as follows.

AWS Message routing rule creation step 3 error action
AWS Message routing rule creation step 3 error action

If you don’t have a Log group, create one using the button Create CloudWatch Log group and select it later. If we have everything set up click the Next button to the review screen and click Create. After saving we should see just created redirection rule in Message Routing console.

AWS IoT Message Routing Console
AWS IoT Message Routing Console

Visualizing data in CloudWatch

The great feature of CloudWatch is ability to create Dashboards and Widgets. That’s why we’ll create a dashboard containing a graph visualizing the air pollution. Let’s go to the CloudWatch console here and click Create dashboard button.

CloudWatch Dashboard creation
CloudWatch Dashboard creation

Select Line widget and metrics as the data source.

CloudWatch Dashboard widget adding
CloudWatch Dashboard widget adding

In the Browse section go to the Air pollution namespace and select bot PM10 and PM2.5. After selecting metrics click Create widget button.

CloudWatch Dashboard selecting metrics for widget
CloudWatch Dashboard selecting metrics for widget

Now our dashboard is almost ready, you only have to save the dashboard using the Save dashboard button at the top. You can monitor how the air pollution level is changing in time.

CloudWatch Dashboard with widget
CloudWatch Dashboard with widget

Creating the CloudWatch Alarm

There are different standards for PM2.5 and PM10 metrics, for my country norms look like:

  • PM2.5 – 25 micrograms per cubic meter of average daily level
  • PM10 – 50 micrograms per cubic meter of average daily level

Let’s try to trigger an alarm when these standards will be exceeded. Navigate to CloudWatch Alarms console here and click Create alarm. In the next step select first metric.

CloudWatch Alarm creation and metric selection
CloudWatch Alarm creation and metric selection

As the Statistic parameter set the Average and as a Period set 1 day.

CloudWatch Alarm creation and setting alarm logic
CloudWatch Alarm creation and setting alarm logic

In the conditions section select the Static Threshold and define the alarm condition as greater than 50. Click Next button.

CloudWatch Alarm creation and setting alarm logic
CloudWatch Alarm creation and setting alarm logic

Now let’s create an SNS topic in which we’ll publish information about Air quality.

CloudWatch Alarm creation and setting an SNS topic
CloudWatch Alarm creation and setting an SNS topic
CloudWatch Alarm review
CloudWatch Alarm review

Set the Alarm name and click Next and then create the Alarm for PM10 metrics. Following the steps above, you need to create a CloudWatch Alarm for the PM2.5 standard which average daily level should be less than 25 micrograms per cubic meter. After that you should see two alarms in CloudWatch Alarms console.

CloudWatch Alarm Console
CloudWatch Alarm Console

Remember that if you are creating the new SNS topic with new endpoint it needs to be verified. You can do this in SNS console here.

Amazon SNS Endpoint verification
Amazon SNS Endpoint verification
Amazon SNS Console
Amazon SNS Console

Now, everything should work and I hope we won’t receive any notifications 😛

Summarize

AWS IoT Core
AWS IoT Core

Thank you for reaching out to that place. I hope you will find my ‘IoT diary’ useful and interesting. I’m starting my journey with IoT so I’m sure some parts of this guide could be done in a better way, so don’t find this as the best guide ever made, however maybe this guide will help you with some similar problems regarding AWS Greengrass and IoT. If you want to know more about AWS and the cloud, check the below posts made by me:

Related Articles

Leave a Comment

This website uses cookies to improve your experience. I'll assume you're ok with this, but you can opt-out if you wish. Read Privacy Policy. Accept