AWS CLI can upload the AWS::Include transform to S3

Introduction

In this post, I'll let you know a new feature introduced in AWS CLI version 1.16.36. Actually, It's just a tiny improvement. But if you use AWS SAM every day in development (like me), it'll make you a little bit happy.

Swagger in AWS SAM

If you use Swagger with AWS SAM to provision Amazon API Gateway, you should use the AWS::Serverless::Api resource. There are two ways to specify Swagger files in the resource:

1. Specifying a Swagger file in an S3 bucket by using the DefinitionUri property
2. Writing an inline Swagger in AWS SAM templates by using the DefinitionBody property

I often choose the second. Because if you use the DefinitionUri property with your Swagger file, AWS SAM treats the file as just a Swagger file, therefore you can't use Intrinsic Functions and Pseudo Parameters in the file. Because there are no specification such features in Swagger. For example, you can't refer to AWS::AccountId and Ref parameters with the Fn::Sub function in Swagger so that you need to write hard-coded AWS Lambda's ARN which specified as an integration request in your Swagger file. In this case, if you divide AWS accounts into multiple environments (production, staging and etc), you may need a bunch of Swagger files for each environment. This is confusing. For these reasons, I prefer the second way rather than the first one.

But, when you write an inline Swagger in AWS SAM templates directly, it'll be very difficult to manage it. Because Swagger and AWS SAM templates are mixed. This is also confusing. To prevent the problem, some developers have hacked the AWS::Include transform. The solution is introduced in AWS SAM's sample project, you might have heard about it.

It's very easy to use. You just need to specify the Fn::Transform in the DefinitionBody property like the following:

ApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
StageName: Sample
DefinitionBody:
Fn::Transform:
Name: AWS::Include
Parameters:
Location: s3:///swagger.yml

If you create a stack by using the file above, the Swagger file specified in the Location property is inserted into the template. In other words, you can divide Swagger files with AWS SAM templates.

Well, what's the matter?

Please look at the Location property in the sample code again. Its value is an S3 path. That is, you need to upload a Swagger file to the S3 path before you create a stack with the template.

Generally, when using AWS SAM, you'll follow the steps below:

• By using aws cloudformation package command, you create a deployment package, upload it to an S3 bucket, and produce a transformed AWS SAM template
• Creating (or updating) a stack with aws cloudformation deploy command

If you followed these steps, you needed to upload a Swagger file to an S3 bucket by yourself. Of course, it's not hard to archive this. You can just use aws s3 cp command. But I was hoping AWS CLI would do it on my behalf. I don't add extra steps (if it's possible) in my deployment pipeline.

But, recently, the upload feature is introduced in AWS CLI version 1.16.36 as the PR is merged. Yeah!

I gave it a shot.

I'll demonstrate the feature to you in this chapter. I tested it in the latest AWS CLI in this time (1.16.37).

$ aws --version
aws-cli/1.16.37 Python/3.6.5 Darwin/17.7.0 botocore/1.12.27

First, I changed the Location property like this:

ApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
StageName: Sample
DefinitionBody:
Fn::Transform:
Name: AWS::Include
Parameters:
Location: src/api/swagger.yml

I just specify a local directory path to the Location property. Then, hit the aws cloudformation package command:

$ aws cloudformation package \
--template-file sam.yml \
--s3-bucket \
--output-template-file template.yml

Uploading to 256b8dbcba1d8ad7f1d4dadda741722d 2371 / 2371.0 (100.00%)
Successfully packaged artifacts and wrote output template to file template.yml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /path/to/template.yml --stack-name

The command seems to be working as I expected. Let's look at the uploaded object in the S3 bucket.

$ aws s3 cp s3:///256b8dbcba1d8ad7f1d4dadda741722d - | md5
256b8dbcba1d8ad7f1d4dadda741722d
$ md5 src/api/swagger.yml
MD5 (src/api/swagger.yml) = 256b8dbcba1d8ad7f1d4dadda741722d

There are no difference between the files. Next, check the transformed AWS SAM template.

ApiGateway:
Properties:
DefinitionBody:
Fn::Transform:
Name: AWS::Include
Parameters:
Location: s3:///256b8dbcba1d8ad7f1d4dadda741722d
StageName: Sample
Type: AWS::Serverless::Api

The value is changed. Its value is an S3 path like the first sample snippet. Therefore, you can deploy it the same as before.

$ aws cloudformation deploy \
--template-file template.yml \
--stack-name

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack -

Conclusion

In this post, a tiny update in AWS CLI is discussed. I think it can be useful in many situations. I hope you find this post helpful.