In this blog post I’ll showcase building of the complete and ready-to-use AWS CDK project to create a simple Chatbot using Amazon Bedrock. I’ll go through the complete design process and also share some considerations at each step. Let’s find out if Serverless and AI can work together as a perfect couple!
Table of Contents
Introduction
With all the hype surrounding AI, I couldn’t resist checking if I could combine my beloved serverless approach with AI. That’s why I decided to do a small project using AWS CDK and python and see what Amazon Bedrock allows us to do.
Let’s start with what were my key assumptions for this project:
- I want to have a user friendly interface that will talk with my Chatbot,
- I want to have the whole solution using Serverless services,
- I want to have a possibility to change the LLM model that I use,
- I want my Chatbot to understand the context of the conversation.
OK, so now we know what we want to build, so let’s start!
Requesting model access in Amazon Bedrock
Before you start, make sure that the model that you want to use for your Chatbot is available for your account. If not, you can request access to it here. In our example we use this model:
Serverless AI Chatbot Architecture
We already know what our assumptions are, now let’s try to transform them into a ready-made solution.
Let’s start with the visual part (client-side). We’ll use the Streamlit library for this. Streamlit lets you transform Python scripts into interactive web apps in minutes. We will use it to provide an interface that communicates with our Chatbot API.
On the backend we will use AWS cloud native services allowing communication with our LLM. We will have an API Gateway that will allow communication with AWS Lambda function that will work as a processor for prompts given by Chatbot users. To connect with Amazon Bedrock we will use a library called LangChain. LangChain helps developers build applications powered by LLMs through a standard interface for models, embeddings, vector stores, and more.
As a caching mechanism, we will use the Amazon ElastiCache Serverless service with the Redis engine.
Serverless AI Chatbot Infrastructure
Using the AWS CDK, let’s now describe the designed infrastructure with the code (IaC). Let’s start by describing the stack into which we will place constructs from API Gateway, ElastiCache, and Lambda functions.
We have here three constructs:
- One that describes the Network (VPC) in which ElastiCache cluster and the Lambda function will reside,
- Second, that describes ElastiCache Serverless cluster itself,
- Third, that describes API Gateway and Lambda function that processes user prompts.
Network Construct
In the network part of our project, we need to create a VPC and an appropriate Security Group allowing connections to the Redis cluster (ElastiCache).
ElastiCache Construct
The previously mentioned ElastiCache Serverless service is relatively new, hence it does not yet have a “more aware” L2 construct, so we have to use the L1 construct.
L1 constructs, also known as CFN resources, are the lowest-level construct and offer no abstraction. Each L1 construct maps directly to a single AWS CloudFormation resource. With L1 constructs, you import a construct that represents a specific AWS CloudFormation resource. You then define the resource’s properties within your construct instance.
In addition to the basic service settings, we need to take care of assigning the appropriate VPC, Security Group and choose the engine used in the service (Redis in our case). Additionally, we export the cluster entry point as CloudFormation Output (CfnOutput) to use it later in the client-side part.
API Gateway and Lambda Construct
Finally, we create a construct for our API. We need to create the appropriate endpoint (in our case /chat/prompt) and connect it to our Lambda function.
We need to allow our Lambda function to work with Bedrock and to connect to our ElastiCache cluster. By following the principle of least privilege (PoLP) let’s create a proper IAM role that will allow that.
With all this in mind, we can define our Lambda function and connect it to serve our chatbot endpoint.
Note that you can also set the different LLM that you’d like your ChatBot to work with. You can set it in constants.py file. In our case we used: meta.llama3-8b-instruct-v1:0.
Serverless AI Chatbot Logic
Now, let’s analyze the logic that stands behind the client (Chatbot interface) and the backend (Lambda function).
Client-side logic
Streamlit will take care of the Chatbot interface itself, while on our side there is still:
- reading and setting the API to which we will send user prompts,
- sending prompts to the proper ChatbotAPI,
- generating responses based on the Chatbot API output,
- assigning users different session IDs so that the Chatbot knows who it is talking to.
In our case, 3 marked elements are responsible for above points.
Backend-side (Lambda) Logic
Our Lambda function needs to set up a connection with caching cluster and also prepare a proper request before sending it to the Bedrock. We need to ask Redis about previous user conversations history, and then if it exists, use it to continue talking with Bedrock instead of starting the new one.
And that’s all 😎 Let’s run it!
Deployment and Demo
You can find a detailed guide on how to start working with my project in README.md file.
Serverless and AI full project
As I promised here’s the link to complete GitHub project, please don’t forget to give it a ⭐ if you find it valuable!
Summarize
This project was initially presented during my session called “Serverless and AI – Can this be the new technological love story?” on AWS Meetup 06/12 – AWS User Group Warsaw.
Thank you for reaching out to that place. If you want to know more about AWS and the cloud, check the below posts:
- How to create Windows EC2 with AWS CDK and Python
- Building latency-based routing solution with Amazon Route 53
- Decrease cost and boost performance by moving to Graviton
- Autoscaling solution for Amazon ECS Cluster
