Automating Image Analysis with AWS Step Functions, Lambda, SQS, SNS, and Rekognition

Automating Image Analysis with AWS Step Functions, Lambda, SQS, SNS, and Rekognition

Clock Icon2024.10.15

Introduction

Hello! I'm Hemanth from the Alliance Department. In this blog, I will walk you through the process of building an automated image analysis workflow using AWS Step Functions, Lambda, SQS, SNS, and Amazon Rekognition. We'll be working with images uploaded to an S3 bucket and using AWS services to identify objects and faces in those images automatically.

AWS

Amazon Web Services, or AWS, is a cloud service platform that provides content distribution, database storage, processing capacity, and other features to support corporate expansion. AWS has offered a broad range of services in many different categories, including Compute, Storage, Networking , Database, Management Tools, and Security.

AWS Lambda

A serverless compute service which runs code as a reply to events and automatically takes care of the bottom resources. It runs code on high availability compute infrastructure and performs all the administration of the compute resources. A few examples are HTTP requests via Amazon API Gateway, changes to objects in S3, and many others.

S3

Simple and popular AWS Service for storage. Replicates data by default across multiple facilities. It charges per usage. It is deeply integrated with AWS Services. Buckets are logical storage units. Objects are data added to the bucket. S3 has a storage class on object level which can save money by moving less frequently accessed objects to a colder storage class.

Amazon Rekognition

Amazon Rekognition is an AWS service that makes it easy to add powerful image and video analysis to your applications. It uses machine learning to detect objects, scenes, activities, and faces, and even recognize celebrities in images.

AWS Step functions

AWS Step Functions is a fully managed service that helps orchestrate the execution of microservices and functions in a workflow. It integrates seamlessly with other AWS services like Lambda, S3, and SQS to automate tasks and build robust applications.

SQS

It is a web service that enables message exchange between software components at any volume without message loss or the need for additional services to be accessible. It contains standard building blocks like cost allocation tags and dead-letter queues and offers a general web services API that may be used through any programming language that the AWS SDK supports.

SNS

Amazon SNS is a highly scalable notification service that allows you to send messages to a large number of subscribers via multiple communication channels, such as email and SMS.

Demo

We will begin by creating two SQS queues: one for processing images containing faces and another for images with objects. Navigate to the AWS Management Console and search for SQS.
Screenshot 2024-10-11 at 16.44.42
Click on "Create Queue."
Screenshot 2024-10-11 at 16.45.26
First queue we will be for images of faces, give queue name and leave the rest of the settings as default. Then click "Create Queue."
Screenshot 2024-10-11 at 16.48.57
Repeat the process to create another queue for images of objects, give a queue name and leave the rest of the settings as default. Then click "Create Queue."
Screenshot 2024-10-11 at 16.48.57
We'll create an SNS topic to send email notifications when a famous personality's image is uploaded to the S3 bucket. Search for "SNS" in the AWS Console and click on "Create Topic."
Screenshot 2024-10-11 at 16.55.45
Select "Standard," give the topic a name, and leave other settings as default.
Screenshot 2024-10-11 at 16.59.14
Screenshot 2024-10-11 at 16.59.45
After creating the topic, click on "Create Subscription" and choose "Email" as the protocol. Enter your email address and confirm the subscription.
Screenshot 2024-10-11 at 17.12.38
Screenshot 2024-10-11 at 17.15.26
Subscription Confirmation email.
Screenshot 2024-10-11 at 17.18.57
Screenshot 2024-10-11 at 17.21.24
Next, create an S3 bucket where images will be uploaded. In the AWS Console, search for "S3" and click on "Create Bucket."
Screenshot 2024-10-11 at 17.03.47
Give a bucket name, keep others as default and click on create bucket.
Screenshot 2024-10-11 at 17.09.32
Screenshot 2024-10-11 at 17.10.59
Now, let's create the workflow to orchestrate the processing of images. In the AWS Console, search for "Step Functions" and click on "Create State Machine."
Screenshot 2024-10-11 at 18.11.21
Choose the "Blank" template and then click "Select."
Screenshot 2024-10-11 at 18.13.43
Select "Code".
Screenshot 2024-10-11 at 18.15.13
Copy the following Step Functions code.
Screenshot 2024-10-11 at 18.22.13
Screenshot 2024-10-11 at 18.24.11
If you go to design you should see the following workflow design.
Screenshot 2024-10-11 at 18.25.57
Now click on config , give a state-machine name, select express as type, let the step function create the role, click on confirm after checking the roles and hit on confirm.
In the state machine, copy the arn.
Screenshot 2024-10-11 at 18.31.47
Now, let's create the Lambda function that will process the images and trigger the Step Function workflow. In the AWS Console, search for "Lambda" and click on "Create Function." Name your function choose Python 3.12 as the runtime, and leave other settings as default.
Screenshot 2024-10-11 at 18.37.31
Paste the following code in the Lambda editor and hit "Deploy".
Screenshot 2024-10-11 at 23.17.14
Screenshot 2024-10-11 at 23.17.51
Screenshot 2024-10-11 at 23.18.23
Screenshot 2024-10-11 at 23.18.54
In configuration select environment variables, then click edit.
Screenshot 2024-10-11 at 23.21.30
Enter the below Key and value the ARN of earlier copied stepfunctions workflow arn and hit save.
Screenshot 2024-10-11 at 23.22.43
Now adding S3 as trigger to our Lambda, click on add trigger.
Screenshot 2024-10-11 at 23.25.30
Give a bucket name, toggle on the acknowledgement and click add. This way, every time a file is uploaded to S3, the Lambda function will be triggered, process the information and send it to Step Functions. However, there is an additional step we need to take before testing.
Screenshot 2024-10-11 at 23.27.15
The Lambda function needs permission to call Step Functions, Rekognition, and S3. Also, the Lambda timeout should be increased as we need to process the image to get the results. First for timeout, go to general configuration and click edit and changing the timeout to 20 seconds to ensure enough time for processing large images.
Screenshot 2024-10-11 at 23.31.26
Screenshot 2024-10-11 at 23.33.06
Go to permissions and click on role.
Screenshot 2024-10-11 at 23.34.28
click on attach policies.
Screenshot 2024-10-11 at 23.36.20
Add all the below shown policies.
Screenshot 2024-10-11 at 23.38.17
First when i upload an image of celebrity and i see stepfunction. In the execution you will get outout of below.
Screenshot 2024-10-11 at 23.43.20
Also corresponding email.
Screenshot 2024-10-11 at 23.44.47
Next when i upload a non celebrity.
Screenshot 2024-10-11 at 23.45.54
Now when i upload an object the output is as below.
Screenshot 2024-10-11 at 23.47.03

Conclusion

By combining AWS Step Functions, Lambda, SQS, SNS, and Rekognition, we've created an automated workflow to analyze images uploaded to an S3 bucket. This workflow can be further expanded by integrating more advanced Rekognition features, custom notifications, or more complex image processing tasks.

Share this article

facebook logohatena logotwitter logo

© Classmethod, Inc. All rights reserved.