AWS CICD: CodeCommit, CodePipeline, CodeBuild, CodeDeploy

CodeDeploy - Primary Components

-Application: Unique name -Compute Platform: EC2/on Prem or Lambda -Deployment Configuration: Deployment rules for success/failuer --e.g. EC2/on Prem: min num of healthy instances --Lambda: how traffic is router to updated lambda version -Deployment Group: (allows to deploy gradually) -Deployment Type: In place or blue/green -IAM instance profile: permissions for EC2 to pull from Github or S3 -Application revision: app code + codespec.yml -Service role: Role for CodeDeploy to perform what it needs -Target version: Target deployment app version

CodeStar Hands On

-Click to create service role -Select Project template(Node.js) -Name your project -You can select CodeCommit or Github for code -Once complete, it will setup basic project and have code in repo, setup pipeline, setup builds

CodeBuild in VPC

-CodeBuild containers run outside your VPC so cannot access VPC resources -You specify: VPC Id, Subnet id, Security Group Id in config then CodeBuild can access(ElastiCache, ec2, ALB, RDS) When? Data query, integration test,

CodePipeline troubleshooting

-Codepipeline state changes happen in AWS cloudwatch Events, which can in return create SNS notifications --you can create events for failed pipelines --you can create events for cancelled stages -If CodePipline fails a stage, your pipeline stops and you can get info in the console -AWS CloudTrail can be used to get AWS API calls -If pipeline cannot perform an action, make sure, "IAM Service Role" attached does have enough permissions

CodePipeline

-Continuous delivery -Visual workflow -Source: GitHub/CodeCommit/S3 -Build:CodeBuild/Jenkins etc -Load Testing: 3rd party tools -Deploy: AWS CodeDeploy/Beanstalk/Cloudformation/ECS -Made of stages: --Each stage can have sequential or parallel actions --Stages examples: Build/Test/Deploy/LoadTest --Manual approval can be defined at any stage

CodeBuild Hands On

-Create Build Project -Give Name of build -Select Repo -Select Branch in the repo -You can select either custom or managed image for Env. Custom in case you have Docker -Select Operating system of the Computes -Service Role you can create separately and select here or let build create its own. It is required to pull code. -Additional Config for: Timeout of Build, Queued Timeout -You can install certs if required. -You can have VPC select if you have a separate VPC -By default buildspec.yml is picked but you can specify if you name the file something else -You can save the Artifact in S3 and also save in cache

CodeDeploy for EC2

-Define using appspec.yml and deployment strategy -Will do in place code deployment for your EC2s -Can use hooks to verify after each deployment phase In place deployment, refer screenshot

CodeDeploy Overview

-Deploy to 100s of EC2 instances which are not managed by Elastic Beanstalk -Open sources tools like Ansible, Puppet, Terraform can be used for deployment -We can use CodeDeploy as it is managed service

Continuous Integration

-Dev push code to repo -A build server checks the code as soon as it is pushed(CodeBuild/Jenkins) -The dev gets feedback about tests and checks that passed or failed -Find bugs early and fix -Deliver faster as code is tested Deploy often

CodeDeploy - Characteristics

-EC2 instances are grouped based on env(dev/test/prod) -Flexibility to define any kind of deployment -Can be chained into CodePipeline and use artifacts from there -Works with any existing tools, with any app, auto scaling integration -Blue/Green deployment works with EC2 only not on prem -Supports AWS lambda deployment -CodeDeploy does not provision resources

CodeDeploy - Steps to make it work

-Each machine(on prem or ec2) has an code deploy agent -Agents keep polling AWS CodeDeploy for work -CodeDeploy sends appspec.yml -App is pulled from S3 or Github -EC2 will run the deployment instructions -CodeDeploy agent will report the status of deployment

AWS CodePipeline Artifacts

-Each pipeline stages can create an artifact -artifacts are passed stored in amazon s3 and passed on the next stage

CodePipeline Hands On

-Enter Name -You will be asked if you want new Service Role or you want to use existing role -You can select Default Artifact Location(S3 will be created) or any other existing S3 which is in same region as pipeline -You can select the Source(CodeCommit/S3/ECR/GitHub/Bitbucket) -You select the Repository Name -Select Branch -Select the Change detection options(AWS CloudWatch(Recommended) or AWS CodePipeline) -Select Build Provider(CodeBuild/Jenkins) -Select Deploy Provider(CloudFormation/Beanstalk/ECS/S3) Selected BeanStalk and the env that was already created there and if you go to Beanstalk, you will see the app getting deployed You can add Stages to your pipeline: Add Stage -Name the Stage -Select Action Provider(e.g. Manual Approval) -You can provide SNS topic where notifications can be sent for Manual approval -You just created one Action Group in a Stage and you can have multiple Action Groups in a Stage -e.g. You can add another Action Group which can Deploy the approved artifact to Prod using BeanStalk -These action groups can be sequential or parallel

CodeCommit Hands on 1

-Go to CodeCommit Console -Create Repo -You can create and push a file there itself -Then you can use menus for Pull requests, Commits, Branches, Git tags. -Under settings, you have Notifications(imp for exam) -Notifications are for any events that happen with your resources. When no automated action is required after event Notification is used. -You create an SNS and notifications will be sent to that SNS -Triggers: when an automated action is required then Triggers

CodeStar

-Integrated solution that regroups: GitHub, CodeCommit, CodeDeploy, CloudFormation, CodePipeline, CloudWatch -Quickly create "CICD-ready" projects for EC2, Lambda, Beanstalk -Supported langs: C#, Go, HTML 5, Java, Node.js, PHP -Issue tracking with Jira and GitHub issues -Ability to integrate with Cloud9 to obtain web IDE(not all regions) -One dashboard to view all components -Free service, pay for underlying -Limited Customization

CodeCommit

-Like Git -private Git repos -No size limit and scales -Fully managed -code in AWS cloud account-secure -Integrates with Jenkins.CodeBuild

CodeBuild Overview

-Managed Service -Alternative to Jenkins -Continuous Scaling(no queues) -Pay for usage-time taken for build -Leverage Docker for reproducible builds -You can use your own Docker images to extend capabilities -Secure --Integration with KMS for build artifact encrypton --IAM for build permissions --VPC for network security --CloudTrail for API call logging -You can pull code from GitHub, CodeCommit, CodePipeline/S3 -Build instructions are in file called buildspec.yml -Output logs of build are in S3 and CloudWatch. So once build finishes the container goes away and all you have is these logs -Metrics to monitor CodeBuild Stats -Use CloudWatch Alarms to detect failed builds and trigger notifs -SNS Notifications -Ability to reproduce CodeBuild locally to troubleshoot in case of errors -Builds can be defined within CodePipeline or CodeBuild itself

CodeCommit Hands on 2

-You can connect to CodeCommit using both SSH and HTTPS -Go to IAM, Users, Select the user you want to allow to access Repo, Select Security Credentials tab, You will see that person's security details -There are 2 options here --SSH keys for AWS CodeCommit: You can put your machines SSH keys here and start cloning the repo with ssh --HTTPS Git credentials to AWS CodeCommit: You can create upto 2 Usernames for this user and you will get UserName and Password. Then go to repo and clone with HTTPS. You come to local machine and pull using HTTPS link and will be asked Username and password.

CodeCommit Notification

-You can trigger notifications in CodeCommit using AWS SNS, AWS Lambda or AWS CloudWatch -Use case for SNS/Lambda: --Deletion of branches --Trigger for pushes that happens in master branch --Notify external Build System --Trigger AWS lambda to perform codebase analysis(creds for committed in code?) -Use case for CloudWatch Event Rules: --Trigger for pull request updates --Commit comment events --CloudWatch Event Rules goes into SNS topic

CodeBuild BuildSpec - What can you do in buildspec file

-buildspec.yml at root of code -Define env variables --Plaintext vars --Secure Secrets: use SSM params store -Phases(commands to run) --Install: install dependencies you may need for your build --Pre Build: final commands just before your build starts --Build: actual build commands --Post Build: finishing touches(zipping file) -Artifacts: What to upload to S3(encrypted with KMS) -Cache: files to cache(dependencies) to S3 for future build speedup

CodeCommit Security

-interacted using standard Git -Authentication in Git: --SSH keys:AWS users can configure SSH keys in their IAM roles --HTTPS: Done through AWS CLI Authentication helper or generating HTTPS credentials -Authorization in Git: --IAM policies manages Users/Roles rights to repos -Encryption: --repos are automatically encrypted at rest using KMS --in transit, using HTTPS or SSH -Cross Account Access: --Dont share SSH keys or AWS creds --use IAM role in your AWS account and use AWS STS(with AssumeRole API)

Continuous Delivery

-reliable software releases whenever needed -Deployments happen often -that means automated deployment --CodeDeploy --Jenkins CD --Spinnaker

CodeDeploy Deployment Config

Config: -One at a time-one instance fails stop deployment -half at a time: 50% -all at once: for dev -custom: min healthy 75% Failures: -Instance stays in failed state -new deployment starts with failed states instance -to rollback, redeploy old deployment or enable auto rollback for failures Deployment Targets: -EC2 instances with tags -Directly to and ASG -Mix of both -customization in script with DEPLOYMENT_GROUP_NAME env variable

CodeDeploy AppSpec: File Section and Hooks

File Section: how to source from Github or S3 into the filesystem Hooks: Set of instructions on how to deploy(have timeouts). Following is the order -ApplicationStop -DownloadBundle -BeforeInstall -AfterInstall -ApplicationStart -VerifyService - very imp

CodeBuild Local Build

For troubleshooting you can run CodeBuild locally after installing Docker For this, use CodeBuild Agent

CodeDeploy for ASG

In Place Deployment: -update existing EC2s -Instances newly created by ASG will get new version Blue Green deployment: -A new ASG created(settings are copied) -set how long to keep old instances -must use an ALB

CodeBuild Supports these environments

Java, Ruby, Python, Go, Node.js, Android, .Net Core, PHP If your language is not there, you can always define your own Docker and extend any env you like

CodeCommit vs GitHub

Similarities: -Both are Git repos -support code reviews -can be integrated with CodeBuild -support HTTPS and SSH Differences: -Security: --GH:GitHub Users --CC:AWS IAM users and roles -Hosted: --GH:by GitHub or Github enterprise on your servers --CC:managed and hosted on AWS -UI: --GH: UI is fully featured --CC: minimal UI

CodeDeploy Hands On

We need 2 roles -CodeDeploy role -EC2 instance role(so that EC2 instance can pull code from S3) Now go back to CodeDeploy -Create application(Name and select platform from EC2/On Prem, Lambda, ECS) -Once app is created, you create DeploymentGroup (set of EC2s where you want to deploy app) -For this first you need some EC2s, go create EC2 with Role that you created above for EC2. Tag this EC2 so that it can be picked up in DeploymentGroup -ssh into the instance, install CodeDeployAgent and run it -Now create DeploymentGroup, Name it, service role will be EC2ServiceRole, DeploymentType(all at once, B/G), Under env Config, you can select the EC2 or on Prem etc and here you will enter that Tag which you assigned to above EC2.(this will create the group) -Select DeploymentSetting as you wish, deploy at once, deploy half, deploy one at a time -Upload code zip into an S3 which will have appspec.yml -Now Create a Deployment, where you will specify app you want to deploy from S3 above. As shown in sample, code was zipped and had appspec.yml file -Once you click Create Deployment, it will start deployment and if everything is good, you can check the EC2 instance address for deployed app.

CodeDeploy Roll Back

You can choose how you want to roll back: -when deployment fails -when alarm threshold is met -do not rollback With rollback, CodeDeploy will rollback to previous good revision and it will create a new deployment

How CodeBuild Works

buildspec.yml and the fact that CodeBuild uses docker is the main thing. Cache Bucket is optional but you can have resolved dependencies there And once build is complete you can cache back