9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

https://jenkins.io/artwork/ by Ksenia Nenasheva

CI/CD Pipeline with Jenkins on AWS

Jenkins CI/CD made easy! Automate your deployment process, save time and reduce risks.
Because automation is a friend, not a nightmare.

Edmar Barros Follow

Feb 1, 2019 · 8 min read

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 1/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

Manual tests and deployments can lead to multiple errors that a well de ned CI/CD
pipeline can avoid while testing your application, inspect the code quality, deploying
and… I could enumerate many others aspects of why CI/CD is a must have but let me
drive you through the setup process instead!

But, what is Jenkins?

Jenkins is a self-contained, open source automation server which can be used to automate
all sorts of tasks related to building, testing, and delivering or deploying software. —
https://jenkins.io/doc/

Jenkins is one of the most popular Open Source CI tools on the market. It o ers support
for multiples SCM and many other 3rd party apps via its plugins. And, with concepts like
Pipeline-as-Code, the entire build process can be checked into a SCM and versioned like
the rest of your code.

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 2/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

Pipe whaat? Pipeline!

… What is a Pipeline?
A continuous delivery (CD) pipeline is an automated expression of your process for getting
software from version control right through to your users and customers.—
https://jenkins.io/doc/book/pipeline/

The Pipeline de nes a set of action that Jenkins will execute in order to test, build and
deploy your application. You can de ne Stages per branches, run commands in parallel,
de ne environment variables and much more.

In this article I’m going to de ne a Pipeline to test and deploy a static web application
into AWS S3.

Setup the build environment

For our build environment we’ll launch an Amazon EC2 instance using the Amazon
Linux 2 AMI, install and con gure the required packages. Make sure that the security
group you select for your instance allows tra c on ports TCP/22 and TCP/80 so you will
be able to connect to you instance via SSH and access it via browser using its DNS.

Install and con gure Jenkins and required packages

Connect to your instance using your private key:

$ ssh -i "path-to-key/key.pem" ec2-user@<instance-name>.<instance-

region>.compute.amazonaws.com

First, let’s update the repositories and install Docker and Git:

$ sudo yum update -y

$ sudo yum install -y docker git

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 3/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

Add the Jenkins repository and install Jenkins from there:

$ sudo wget -O /etc/yum.repos.d/jenkins.repo

http://pkg.jenkins.io/redhat/jenkins.repo
$ sudo rpm — import https://pkg.jenkins.io/redhat/jenkins.io.key
$ sudo yum install jenkins -y
$ sudo usermod -aG root jenkins # Add jenkins to the root group

To allow Jenkins to build our Docker images, we need to add the jenkins user to the
docker group:

# Add Jenkins and current user to the Docker group

$ sudo usermod -aG docker $USER
$ sudo usermod -aG docker jenkins

Add the Jenkins and Docker services to start on boot and start both services:

# Add Jenkins and Docker to startup

$ sudo chkconfig jenkins on
$ sudo chkconfig docker on
$ sudo chkconfig --list

# Start Jenkins and Docker as a service

$ sudo service jenkins start
$ sudo service docker start

As Jenkins typically uses port TCP/8080, we’ll con gure iptables routing rules to redirect
the tra c:

$ sudo iptables -I INPUT 1 -p tcp --dport 8080 -j ACCEPT

$ sudo iptables -I INPUT 1 -p tcp --dport 80 -j ACCEPT

$ sudo iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j

REDIRECT --to-port 8080

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 4/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

I’m not covering the usage of SSL certi cates, to use HTTPS you need to include rules to
redirect tra c for port 443:

$ sudo iptables -I INPUT 1 -p tcp --dport 8443 -j ACCEPT

$ sudo iptables -I INPUT 1 -p tcp --dport 443 -j ACCEPT

$ sudo iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 443 -j

REDIRECT --to-port 8443

You can nd more about SSL certi cates on this article from Paul Lessing:

Single-Page Apps on AWS, Part 1: Hosting a Website on S3

How to set up S3, CloudFront and CloudFlare to host a static website on S3
without a dedicated server.

medium.com

At this point you should be able to see the Jenkins home page using the public DNS
name of your EC2 instance(e.g. ec2–<ec2-public-ip-address>.compute-
1.amazonaws.com):

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 5/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

Jenkins Setup — Administrator password

Get the password from the indicated le:

$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Install the recommended plugins and create an Admin account in the following steps.

At this point you should be able to login an see the following page:

Jenkins home page

Install Jenkins Plugins

At the Jenkins home page on the left menu select Manage Jenkins -> Manage Plugins
select the tab Available and search for the following plugins:

Blue Ocean - New Jenkins UI

Pipeline AWS - AWS Integration

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 6/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

Create the Pipeline AWS Jenkins Credential

At the Jenkins home page on the left menu click at Credentials->System, select the scope
global and at the left menu again Add Credential:

Create AWS Credential

Provide the AWS IAM Credentials to allow Jenkins Pipeline AWS Plugin to access your
S3 Bucket. At the above image, insert the created Access Key ID and the Secret Access Key.
You can de ne and ID that will be used at the Jenkins le con guration as the following
example:

withAWS(region:'<your-bucket-region>',credentials:'<Jenkins-
Credential-ID-AWS>') {

The <Jenkins-Credential-ID> is the ID of the credential you’ve just created.

Blue Ocean Pipeline

Jenkins Blue Ocean UI makes it easier to create and con gure a new pipeline.

1. Create a new pipeline and select GitHub as repository

2. Enter a Personal Access Token from GitHub to allow Jenkins to access and scan your
repositories.

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 7/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

3. Follow the instructions and the Blue Ocean will look for a Jenkins le, if does not
exists Blue Ocean allows you to de ne a new le and con gure the Pipeline.

If you create a Pipeline with Blue Ocean it will commit the Jenkins le to your repository
and it will have a structure similar to the following:

1 pipeline {
2 agent {
3 docker {
4 image 'node:10-alpine'
5 args '-p 20001-20100:3000'
6 }
7 }
8 environment {
9 CI = 'true'
10 HOME = '.'
11 npm_config_cache = 'npm-cache'
12 }
13 stages {
14 stage('Install Packages') {
15 steps {
16 sh 'npm install'
17 }
18 }
19 stage('Test and Build') {
20 parallel {
21 stage('Run Tests') {
22 steps {
23 sh 'npm run test'
24 }
25 }
26 stage('Create Build Artifacts') {
27 steps {
28 sh 'npm run build'
29 }
30 }
31 }
32 }
33 stage('Deployment') {
34 parallel {
35 stage('Staging') {
36 when {
37 branch 'staging'
https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 8/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

38 }
39 steps {
40 withAWS(region:'<your-bucket-region>',credentials:'<AWS-Staging-Jenkins-Credential-ID
41 s3Delete(bucket: '<bucket-name>', path:'**/*')
42 s3Upload(bucket: '<bucket-name>', workingDir:'build', includePathPattern:'**/*');
43 }
44 mail(subject: 'Staging Build', body: 'New Deployment to Staging', to: 'jenkins-maili
45 }
46 }
47 stage('Production') {
48 when {
49 branch 'master'
50 }
51 steps {
52 withAWS(region:'<your-bucket-region>',credentials:'<AWS-Production-Jenkins-Credentia
53 s3Delete(bucket: '<bucket-name>', path:'**/*')
54 s3Upload(bucket: '<bucket-name>', workingDir:'build', includePathPattern:'**/*');
55 }
56 mail(subject: 'Production Build', body: 'New Deployment to Production', to: 'jenkins
57 }
58 }
59 }
60 }
61 }
62 }

Jenkinsfile hosted with ❤ by GitHub view raw

Jenkins le pipeline example

The above Jenkins le has a pipeline as you can see at the following image:

Jenkins Blue Ocean Pipeline

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 9/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

The Pipeline above uses a docker agent to test the application and to create the nal
build les. The Deployment stage will remove the les from a given S3 bucket, upload
the new build les and send an email to inform that a new deployment was successfully
completed.

One thing to note is at the agent de nition it used a port range is used to avoid the job to
fail in case a given port is already in use

Setup the GitHub Webhook

A WebHook is an HTTP callback: an HTTP POST that occurs when something happens;
a simple event-noti cation via HTTP POST. A GitHub Webhook allows Jenkins to
subscribe to repository events like Pushes and Pull Requests.

The Webhook will send a POST request to your Jenkins instance to notify that an action
was triggered(PR, Push, Commit…) and it should run a new job over the code changes.

Go to you repository Settings, select Webhooks on the left menu and click on the
button Add webhook.

GitHub Repository Settings — Webhooks

Set the Payload URL to http://<your-jenkins-instance>/github-webkook/ and the

Content type to application/json

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 10/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

GitHub Repository Settings — Create a Webhook

Select the individual events that you want to be used to trigger a new Jenkins build. I
recommend to select Pushes and Pull Requests.

After that you will be able to see a similar message after a PR is created and it passes
Jenkins validation:

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 11/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

GitHub Pull Request Status

Create the S3 Bucket

At your AWS console go to Services -> S3 and click the Create bucket button. Choose
you bucket name and region, untick both options under the Manage public bucket

polices for this bucket and nish the bucket creation. Open the bucket Properties, turn
on the Static website hosting and de ne the index.html to point to your entry le like the
following image:

AWS S3 Bucket Properties — Static website hosting

To use a custom domain name you need to create a second bucket and under Static
website hosting select the Redirect requests option. That option allows you to de ne the
domain that will be used to access the hosting bucket. Bear in mind that this secondary
bucket will use a HTTP status code 301 to redirect the requests.

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 12/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

At this point you should be able to access your website using you custom domain you via
the bucket

http://<bucket-name>.s3-website-<region>.amazonaws.com

Conclusion
This process has multiple pros and cons and I would like to drop my two cents on this
and highlight a couple of them.

Pros

Cheaper process with a good performance of a CDN to deliver static content.

AWS handles the tra c so scalability won’t be an issue.

Cons

Cache! Updates may take a while to be propagated as it has to wait the cache to
expire.

Server side rendering. Bots and crawler won’t be able to get any metadata.

Despite the cache, that can be handled by setting to the index.html a short maxAge, this
solution can be considered robust and sustainable as it takes advantages of multiple
AWS resources to make its availability and scalability to follow high standards as it is
required nowadays.

. . .

Thanks for reading!

Feel free to reach me out on Twitter, Linkedin, and here on Medium.

References

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 13/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

Lessing, Paul. ‘Single-Page Apps on AWS, Part 1: Hosting a Website on S3’. Medium, 16
Apr. 2018, https://medium.com/@P_Lessing/single-page-apps-on-aws-part-1-hosting-
a-website-on-s3-3c9871f126.

‘Personal API Tokens’. The GitHub Blog, 17 May 2013, https://github.blog/2013-05-16-

personal-api-tokens/.

‘Pipeline’. Pipeline, https://jenkins.io/doc/book/pipeline/index.html. Accessed 28 Jan.

2019.

‘Pipeline as Code’. Pipeline as Code, https://jenkins.io/doc/book/pipeline-as-

code/index.html. Accessed 28 Jan. 2019.

Solution 1: Vanilla CI/CD Pipeline — Jenkins on AWS.

https://docs.aws.amazon.com/aws-technical-content/latest/jenkins-on-aws/solution-
1-vanilla-cicd-pipeline.html. Accessed 28 Jan. 2019.

‘Webhooks’. GitHub Developer, https://developer.github.com/webhooks/ Accessed 28

Jan. 2019.

. . .

Learn more
Code, Quick. ‘Top Tutorials To Learn Jenkins CI For Testing Automation’. Medium, 4
June 2018, https://medium.com/quick-code/top-tutorials-to-learn-jenkins-ci-for-
testing-automation-93c7ac068f66.

Join our community Slack and read our weekly Faun topics ⬇

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 👏 14/15
9/22/2020 CI/CD Pipeline with Jenkins on AWS | by Edmar Barros | FAUN | Medium

If this post was helpful, please click the clap 👏 button below a few times to
show your support for the author! ⬇

Thanks to Adriano Vinhas and Paulo Santos.

Jenkins AWS Cicd S3 Software Engineering

About Help Legal

Get the Medium app

https://medium.com/faun/ci-cd-pipeline-with-jenkins-and-aws-s3-c08a3656d381 15/15