2.7K

IoT (eng. Internet of Things) is becoming the most important approach in creating consumer and industrial solutions. In this guide we’ll try to successfully connect the IoT device with cloud. To do that, we’ll build an air pollution monitoring system which will notify us about our air quality, all of that will be achieved thanks to NanoPi Neo V1.4, SDS011 sensor, AWS IoT, AWS Greengrass, AWS CloudWatch and Amazon SNS.

Prerequisite

Before we start you need to make sure that you have the following:

• AWS account
• NanoPi Neo V1.4 or similar single board computer (SBC)
• SDS011 sensor + UART-USB connector

IoT project overview

What we’ll try to build is already mentioned air pollution monitoring system. NanoPi device will be connected by UART-USB connector with SDS011 sensor, which measures PM2.5 and PM10 markers of air pollution. NanoPi will be connected to the AWS IoT core by AWS Greengrass. Using AWS Greengrass components we’ll deploy at edge, simple Python app which will collect and publish (using AWS Greengrass MQTT component) air pollution metrics to the cloud. Using the AWS IoT Message Routing we’ll send measured data to the AWS CloudWatch, and there we’ll create a CloudWatch Alarm connected to SNS topic. Alarm will be triggered when air pollution norms will be exceeded.

The connection between device and sensor should look similar to the below image:

Ubuntu 20.04 installation on NanoPi

The first what we need to do is installing Linux software on our IoT device. I’ve created a separate post mentioning how to install Ubuntu 20.04 on NanoPi Neo V1.4 in a ‘headless’ mode. Check the link here. Next, we need to ensure that we have below things installed in order to run Greengrass on the device:

• Python 3.8+ with pip (guide is located here)
• cmake, curl, zip, unzip, git
• Java 11 / AWS Correto 11 (if you find your device incompatible with Correto, use OpenJDK made by Azul)

IAM User creation

First of all we need to setup IAM user which will be used for performing Greengrass operations. Follow the link to go to the AWS IAM console.

Create a user with Access key – Programmatic access and assign him AdministratorAccess policy (WARNING! We use admin access only because of the fact that we are creating development environment and we’d like to ensure that lack of permission won’t disturb our workshops. Don’t use AdministratorAccess policy in production environment!).

Copy the Access key ID and Secret access key and save them as environment variables inside NanoPi device and local device using the following commands:

export AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<AWS_SECRET_ACCESS_KEY>

Installing AWS IoT Greengrass Development Kit Command-Line on local computer

Our Python app is a Greengrass component built to collect data from SDS011 sensor. Thanks to the Greengrass we’ll build the component on our local computer, then using the AWS IoT Greengrass Development Kit Command-Line (GDK) we’ll publish this component code to S3 bucket and after that we’ll run this component on our NanoPi device using the AWS Console only.

First of all, let’s download to our local computer my Python app which purpose is to monitor air pollution and connect using MQTT with AWS IoT:

git clone https://github.com/amswiatkowski/air_pollution_iot_metter.git

Next, we need to build our artifact in order to publish it to an S3 bucket. To do this we need AWS IoT GDK. Let’s install it:

pip install -U git+https://github.com/aws-greengrass/[email protected]

If you’ll find any problems at this point, please take a look at developer’s guide here.

Type below command to check if everything is working as expected:

gdk --version

You should see similar view:

Next, let’s jump to the directory where we’ve cloned our project code and install all Python requirements.

pip install -r requirements.txt

AWS Greengrass project structure

After installing all of the requirements, we can proceed with building and publishing artifact to the S3 bucket, but first let’s take a quick look how to project looks like:

gdk-config.json is a file where we configure how the building of component should work. In our project, we are telling GDK that we want to have a zip file which will be published to the S3 bucket prefixed with iot-greengrass-air-pollution. NEXT_PATCH is used to automatically increase the version number of our package by GDK.

gdk-config.json:

{
  "component": {
    "com.cloudybarz.airPollutionMetter": {
      "author": "sz3jdii",
      "version": "NEXT_PATCH",
      "build": {
        "build_system": "zip"
      },
      "publish": {
        "bucket": "iot-greengrass-air-pollution",
        "region": "eu-central-1"
      }
    }
  },
  "gdk_version": "1.0.0"
}

recipe.json is a file that defines a component’s details, dependencies, artifacts and lifecycles. In the ComponentConfiguration section we define a rule to allow the component to publish to the IoT MQTT topic. In Manifests section we define steps which our component needs to do on each selected platform. The component Lifecycle specifies the commands to run, install and shut down the component. Artifacts section defines what files are needed by our component this section also defines location of them.

recipe.json:

{
  "RecipeFormatVersion": "2020-01-25",
  "ComponentName": "com.cloudybarz.airPollutionMetter",
  "ComponentVersion": "COMPONENT_VERSION",
  "ComponentDescription": "Simple air pollution metter built with AWS Greengrass.",
  "ComponentPublisher": "Sz3jdii",
  "ComponentDependencies": {
  },
  "ComponentConfiguration": {
    "DefaultConfiguration": {
      "accessControl": {
        "aws.greengrass.ipc.mqttproxy": {
          "com.cloudybarz.airPollutionMetter:mqttproxy:1": {
            "policyDescription": "Allows access to publish/subscribe to all topics.",
            "operations": [
              "aws.greengrass#PublishToIoTCore",
              "aws.greengrass#SubscribeToIoTCore"
            ],
            "resources": [
              "*"
            ]
          }
        }
      }
    }
  },
  "Manifests": [
    {
      "Platform": {
        "os": "linux"
      },
      "Lifecycle": {
        "Install": {
          "RequiresPrivilege": true,
          "script": "pip3 install -r {artifacts:decompressedPath}/air_pollution_iot_metter/requirements.txt"
        },
        "Run": {
          "RequiresPrivilege": true,
          "script": "python3 -u {artifacts:decompressedPath}/air_pollution_iot_metter/main.py"
        }
      },
      "Artifacts": [
        {
          "URI": "s3://iot-greengrass-air-pollution/air_pollution_iot_metter.zip",
          "Unarchive": "ZIP",
          "Permission": {
            "Read": "ALL"
          }
        }
      ]
    }
  ]
}

The /src directory contains the logic of the application. Now, as you know what the structure of AWS Greengrass components looks like, we can build and upload our component to an S3 bucket. First of all we need to assign a policy which allows access to an S3 bucket by the Greengrass execution role. Create a file named: component-artifact-policy.json with the below content.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::iot-greengrass-air-pollution*/*"
    }
  ]
}

Run the below command on the local machine:

aws iam create-policy \
  --policy-name MyGreengrassV2ComponentArtifactPolicy \
  --policy-document file://component-artifact-policy.json

Copy the policy Amazon Resource Name (ARN) from the policy metadata in the output. You will use this ARN to attach this policy to the NanoPi Greengrass device role in the next step. Replace the REPLACE_WITH_ARN with the ARN copied and run the below command:

aws iam attach-role-policy \
  --role-name GreengrassV2TokenExchangeRole \
  --policy-arn arn:aws:iam::REPLACE_WITH_ARN:policy/MyGreengrassV2ComponentArtifactPolicy

If the command has no output, it succeeded.

Now, let’s build and publish our component. From the directory of the project on our local machine run the below commands:

gdk component build
gdk component publish

If everything went well, you should see below output:

You can also check if an S3 bucket was created inside AWS console.

Connecting NanoPi to the cloud using AWS Greengrass

If we have everything setup well, we can proceed with connecting our IoT setup with the cloud. Let’s jump into the AWS IoT Greengrass console here.

Follow the “Set up one core device” link and fill the form with our device information.

Let’s run both commands on the NanoPi device (login to it using SSH) to download and install Greengrass components.

If everything went well, we should see our device registered in the Greengrass core devices list.

Deploying the component to the IoT device

Let’s summarize what we should achieve for now. We have set up a connection between the IoT device (NanoPi + SDS011) and AWS IoT using the Greengrass. Thanks to the Greengrass GDK we’ve also created our first component which now waits on an S3 bucket for being deployed to the IoT core device. To do that, let’s jump to the Deployments section in the AWS Greengrass console here.

Check the box near the name of the default deployment and click Revise button.

Leave the Step 1 form as it is. Click the Next button to go to the next step.

Select the component which we’ve published using GDK from our local machine. After that, click the Next button and go straight to the Review Step (Step 5).

Lastly, click the Deploy button and wait a little.

Logs and monitoring for IoT component

If you want to monitor the status of the component and device you can focus on below two files located in IoT device. com.cloudybarz.airPollutionMetter.log represents the state and outputs from the components which we’ve just deployed.

tail -f /greengrass/v2/logs/com.cloudybarz.airPollutionMetter.log 

greengrass.log file represents the state of Greengrass services running on the IoT device.

tail -f /greengrass/v2/logs/greengrass.log

Subscribing to MQTT topic

If our deployment went well, we can use builtin MQTT test client located in the AWS IoT console here. Our Python app, publishes to the #pollution topic metrics collected from the sensor. Let’s subscribe to that topic to see if we can process them later.

After a while we should see messages coming to the client which will mean that everything is working fine.

Routing the MQTT messages to the CloudWatch service

Now, as we have data coming properly to the cloud, we’d like to visualize it in the form of a graph and notify user if air pollution will be exceeded. There’s a great AWS IoT feature called Message Routing, which can allow us to redirect MQTT messages to other AWS service such as AWS CloudWatch. The great feature of Message Routing is also creating redirection rules based on an SQL query. Let’s jump to the Message routing console here. Click the Create rule button and name the rule “Publish_air_pollution_metrics_to_Cloudwatch”.

Click the Next button to go to the Step 2. In this step we need to set an SQL query to grab only MQTT messages with selected conditions and from selected topic. In our case we want to forward every message sent to the ‘pollution’ MQTT topic so we can use below query.

SELECT * FROM 'pollution'

Click the Next button and go to the Step 3 where we’ll set actions. We need to set two actions, one for each metric. Let’s see how properly fill the forms.

If you don’t have IAM role, create one using the Create new role button. Let’s setup 2. action similar to the first one.

We want also to know if something goes wrong so we can set up an error action which will store an error message in Cloudwatch Logs. In the same form as above, fill the error action fields as follows.

If you don’t have a Log group, create one using the button Create CloudWatch Log group and select it later. If we have everything set up click the Next button to the review screen and click Create. After saving we should see just created redirection rule in Message Routing console.

Visualizing IoT data in CloudWatch

The great feature of CloudWatch is ability to create Dashboards and Widgets. That’s why we’ll create a dashboard containing a graph visualizing the air pollution. Let’s go to the CloudWatch console here and click Create dashboard button.

Select Line widget and metrics as the data source.

In the Browse section go to the Air pollution namespace and select bot PM10 and PM2.5. After selecting metrics click Create widget button.

Now our dashboard is almost ready, you only have to save the dashboard using the Save dashboard button at the top. You can monitor how the air pollution level is changing in time.

Creating the CloudWatch Alarm

There are different standards for PM2.5 and PM10 metrics, for my country norms look like:

• PM2.5 – 25 micrograms per cubic meter of average daily level
• PM10 – 50 micrograms per cubic meter of average daily level

Let’s try to trigger an alarm when these standards will be exceeded. Navigate to CloudWatch Alarms console here and click Create alarm. In the next step select first metric.

As the Statistic parameter set the Average and as a Period set 1 day.

In the conditions section select the Static Threshold and define the alarm condition as greater than 50. Click Next button.

Create an SNS topic

Now let’s create an SNS topic in which we’ll publish information about Air quality.

Next, set the Alarm name and click Next and then create the Alarm for PM10 metrics. After following the steps above, you need to create a CloudWatch Alarm for the PM2.5 standard which average daily level should be less than 25 micrograms per cubic meter. After that you should see two alarms in CloudWatch Alarms console.

Remember that if you are creating the new SNS topic with new endpoint it needs to be verified. You can do this in SNS console here.

Now, everything should work and I hope we won’t receive any notifications 😛

Summarize

Thank you for reaching out to that place. I hope you will find my ‘IoT diary’ useful and interesting. I’m starting my journey with IoT so I’m sure some parts of this guide could be done in a better way, so don’t find this as the best guide ever made, however maybe this guide will help you with some similar problems regarding AWS Greengrass and IoT. If you want to know more about AWS and the cloud, check the below posts made by me:

• Canary deployment with Linkerd and Kubernetes
• How to run your own mail server on Amazon AWS EC2
• Headless setup of Ubuntu on NanoPi NEO