Streamlining Lambda Deployments with CodePipeline, CodeBuild, CloudFormation, and Multiple Lambda Functions
Developing and deploying serverless applications with AWS Lambda functions can be a smooth process with the right tools. However, managing multiple Lambda functions within a single project can become complex, especially when needing to deploy updated code for each function. This article explores how to use AWS CodePipeline, CodeBuild, CloudFormation, and Lambda to build and deploy multiple Lambda functions within a single pipeline, ensuring efficient code deployment and management.
The Challenge: Managing Multiple Lambdas
Imagine a scenario where you're building a serverless application that requires several Lambda functions for different tasks, each with its own codebase. Manually deploying each Lambda function individually can be time-consuming and error-prone. We need a solution to automate this process, ensuring consistent and efficient deployments while maintaining code organization and clarity.
The Solution: A Streamlined Pipeline
Here's how we can achieve this streamlined workflow:
1. Project Structure:
my-project
├── lambda1
│ ├── index.js
│ └── package.json
├── lambda2
│ ├── index.js
│ └── package.json
├── cloudformation
│ └── template.yaml
└── pipeline.yaml
- lambda1, lambda2: Directories containing the code for each Lambda function, including the entry point (
index.js) and its dependencies (package.json). - cloudformation: Contains the CloudFormation template (
template.yaml) defining the resources for your Lambdas, including their configurations and triggers. - pipeline.yaml: Defines the CodePipeline configuration, outlining the build and deployment steps.
2. CodeBuild Buildspec:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 16
commands:
- npm install -g aws-sdk
- cd lambda1
- npm install
- cd ../lambda2
- npm install
- cd ..
build:
commands:
- cd lambda1
- npm run build
- cd ../lambda2
- npm run build
- cd ..
- aws cloudformation package \
--template-file cloudformation/template.yaml \
--output-template-file cloudformation/packaged.yaml \
--s3-bucket <YOUR_S3_BUCKET_NAME> \
--s3-prefix my-project-stack
- aws s3 cp lambda1/build/* s3://<YOUR_S3_BUCKET_NAME>/my-project-stack/lambda1/
- aws s3 cp lambda2/build/* s3://<YOUR_S3_BUCKET_NAME>/my-project-stack/lambda2/
post_build:
commands:
- echo "Build complete!"
- This buildspec installs necessary dependencies, builds the code for each Lambda function, packages the CloudFormation template, and uploads the built code and the template to your S3 bucket.
3. CloudFormation Template:
Resources:
Lambda1:
Type: 'AWS::Serverless::Function'
Properties:
Handler: index.handler
Runtime: nodejs16.x
CodeUri: s3://<YOUR_S3_BUCKET_NAME>/my-project-stack/lambda1/
...
Lambda2:
Type: 'AWS::Serverless::Function'
Properties:
Handler: index.handler
Runtime: nodejs16.x
CodeUri: s3://<YOUR_S3_BUCKET_NAME>/my-project-stack/lambda2/
...
- The CloudFormation template defines the Lambda functions, including their configurations, code locations (pointing to the S3 bucket), and any other necessary resources.
4. CodePipeline Pipeline:
version: 2015-07-09
Resources:
LambdaDeploymentPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: LambdaDeploymentPipeline
RoleArn: arn:aws:iam::<YOUR_AWS_ACCOUNT_ID>:role/<YOUR_CODE_PIPELINE_ROLE>
ArtifactStore:
Type: S3
Location: <YOUR_S3_BUCKET_NAME>
Stages:
- Name: Source
Actions:
- Name: Source
ActionTypeId:
Category: Source
Owner: AWS
Provider: CodeCommit
Version: 1
Configuration:
RepositoryName: my-project
BranchName: main
- Name: Build
Actions:
- Name: Build
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: <YOUR_CODEBUILD_PROJECT_NAME>
InputArtifacts:
- Name: SourceArtifact
OutputArtifacts:
- Name: BuildArtifact
- Name: Deploy
Actions:
- Name: Deploy
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: 1
Configuration:
ActionMode: CHANGE_SET_REPLACE
StackName: my-project-stack
TemplatePath: s3://<YOUR_S3_BUCKET_NAME>/my-project-stack/packaged.yaml
InputArtifacts:
- Name: BuildArtifact
- This CodePipeline definition defines the pipeline stages: Source (fetching code from CodeCommit), Build (using CodeBuild to build the code), and Deploy (using CloudFormation to deploy the Lambda functions and their configurations).
5. Deployment Process:
- Source: CodePipeline fetches the latest code from your CodeCommit repository.
- Build: CodeBuild executes the buildspec, installing dependencies, building the code, packaging the CloudFormation template, and uploading the build artifacts to your S3 bucket.
- Deploy: CloudFormation uses the uploaded template and code to provision the resources (Lambda functions, etc.) and deploy the updated code.
Benefits of this Approach
- Simplified Code Management: Organize Lambda function code in separate directories for better code organization and maintainability.
- Automated Deployments: Eliminate manual deployments, saving time and reducing errors.
- Streamlined Pipeline: Centralize the deployment process, allowing consistent deployments across all Lambda functions.
- Version Control: Code changes are tracked in your source control system, allowing easy rollback and history tracking.
- Scalability: Easily adapt to growing application requirements by adding new Lambda functions to your pipeline.
Conclusion
By leveraging the power of CodePipeline, CodeBuild, CloudFormation, and Lambda, you can efficiently build, deploy, and manage multiple Lambda functions within a single, robust pipeline. This approach promotes code organization, automates deployments, and provides a streamlined workflow for building and deploying serverless applications. Remember to adapt these examples to your specific needs, including choosing the appropriate deployment tools and configuring the pipeline based on your project requirements.
Resources
This streamlined approach to managing multiple Lambdas provides a solid foundation for developing and deploying complex serverless applications. By implementing this framework, you can significantly enhance your development efficiency and reduce deployment complexities.
