> For the complete documentation index, see [llms.txt](https://tkssharma-devops.gitbook.io/devops-training/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://tkssharma-devops.gitbook.io/devops-training/devops-01-continuous-integration/continuous-integration-and-continuous-delivery/ci-cd-using-gitlab-ci-s3/ci-integration-with-aws/ci-integration-with-ec2.md).

# CI Integration with EC2

We are going to achieve following things in this topic, we will setup CI/CD for node JS application which will deploy application to EC2 Instance 

* We will have node js gitlab repository
* Will  run `lint` and `test` on every branch of our code (Continuous Integration)
* If everything passes in previous step, tell GitLab to deploy these changes to AWS EC2 instances only for **master branch**(Continuous Deployment)

#### Continuous Integration with GitLab <a href="#id-6ef9" id="id-6ef9"></a>

[GitLab has excellent support for CI](https://about.gitlab.com/features/gitlab-ci-cd/). To enable this, we need to create `.gitlab-ci.yml` file.

```
# Node docker image on which our code would run
image: node:8.9.0

#This command is run before all the jobs
before_script:
  - npm install

stages:
  - test

# lint and test are two different jobs in the same stage.
# This allows us to run these two in parallel and making build faster

# Job 1:
lint:
  stage: test
  script:
    - npm run lint
# Job 2:
test:
  stage: test
  script:
    - npm run test
```

Lets understand this file.

*Line 2:* Tells which Docker image to use

*Line 5,6:* Run npm install before any job

*Line 8–9:* We have created only 1 stage with a name `test` , could be any name

*Line 15–18:* We have created a job with a name `lint` . Again this could be any name. We want this job to run in a test stage. We want it to run `npm run lint`command. `lint` script is defined in our `package.json` file.

*Line 20–23:* Same as previous job but this time, run `npm run test` command

Since we have not provided any branch, this would run on all the branches whenever a commit is made. Important thing to note is that all the jobs ( `lint` and `test`) will run in parallel as they are part of the same stage. If you want them to run sequentially, you can write multiple command inside `script` tag.

#### **Continuous Deployment with GitLab** <a href="#fa23" id="fa23"></a>

We need to do three steps for deployment

* Update our GitLab config to prepare our docker image
* Enable GitLab container to ssh into our remote servers
* Take the latest changes from GitLab and restart server

#### Update .gitlab-ci.yml <a href="#id-1341" id="id-1341"></a>

Lets update our `.gitlab-ci.yml` file to reflect our new requirement

```
# Node docker image on which this would be run
image: node:8.9.0

#This command is run before actual stages start running
before_script:
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  - npm install

stages:
  - test
  - deploy

# lint and test are two different jobs in the same stage.
# This allows us to run these two in parallel and making build faster

# Job 1:
lint:
  stage: test
  script:
    - npm run lint

# Job 2:
test:
  stage: test
  script:
    - npm run test
    # ToDo: Add coverage

deployToAWS:
  only:
    - master
  stage: deploy
  script:
    - bash deploy/deploy.sh
```

We have made three changes in the above file.

* *Line 6:* We want to make sure our instance has `openssh-client` installed. More on this later.
* *Line 11:* We have added another stage called `deploy` . It will only run after successful completion of `test` stage.
* *Line 29–34:* We would like to run this on master branch only. We want it to run a script located at `deploy/deploy.sh`

#### Enable GitLab container to ssh into AWS EC2 instance <a href="#id-7aee" id="id-7aee"></a>

We need to solve for two problems:

* ***How to give `ssh` key and server information to GitLab docker container so that it can ssh into EC2 instance:*** If we want to ssh into our AWS EC2 instance from our local machine, we do something like this

```
ssh -i myPemFile.pem ubuntu@<ip address of the instance>
```

It’s not a good idea to commit the key in our source code even if its private repo. So we will keep it in GitLab as an environment variable. Go to “Settings => CI/CD” and create two variables:

![](https://cdn-images-1.medium.com/max/1600/1*YcQubJ6MEvss7R3_8zgzFQ.png)

```
PRIVATE_KEY <Insert the key you downloaded when creating an EC2 instance>
```

```
DEPLOY_SERVERS <comma separated ip values of all the servers to which you want to deploy this code>
```

These variables would be available to us in our docker container as environment variables. We installed `ssh-agent` in our GitLab docker container so that we can ssh into the EC2 instance. Lets start the ssh agent and add this key to this docker image.

```
eval $(ssh-agent -s)
echo "$PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
```

GitLab at times add spaces which will fail `ssh` into the EC2 instance and hence the `tr -d ‘\r'` . Issue [here](https://gitlab.com/gitlab-examples/ssh-private-key/issues/1).

* ***How to disable the prompt for `hostKeyChecking:`*** Whenever we ssh into a EC2 instance that we have never sshed before, we get a prompt like this

host key checking when we ssh into the instance for the first time

![](https://cdn-images-1.medium.com/max/1600/1*fkZ6V1Y39ofIYrWkOHZ-qw.png)

We need to disable this as it would stop our deployment.

To disable this, we need to create a entry in \~`/.ssh/config` to looking something like this

```
Host *
    StrictHostKeyChecking no
# This the the prompt we get whenever we ssh into the box and get the message like this
#
# The authenticity of the host 'ip address' cannot be verified....
#
# Below script will disable that prompt

# note ">>". It cretes a file if it does not exits.
# The file content we want is below
#
# Host *
#   StrictHostKeyChecking no
#

# any future command that fails will exit the script
set -e
mkdir -p ~/.ssh
touch ~/.ssh/config
echo -e "Host *\n\tStrictHostKeyChecking no\n\n" >> ~/.ssh/config
```

Make sure this file is executable. We can change the permission with `chmod a+x` during commit and during our deploy script.

Now that we have solved both the problems, all we need is a bash script which can loop through the all the ips, ssh into each one of them and execute the script

```
DEPLOY_SERVERS=$DEPLOY_SERVERS // We defined this in GitLab
ALL_SERVERS=(${DEPLOY_SERVERS//,/ })
for server in "${ALL_SERVERS[@]}"
do
  ssh ubuntu@${server} 'bash' < ./deploy/updateAndRestart.sh
done
```

So our full `deploy.sh` would look like this

```
#!/bin/bash

# any future command that fails will exit the script
set -e

# Lets write the public key of our aws instance
eval $(ssh-agent -s)
echo "$PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null

# ** Alternative approach
# echo -e "$PRIVATE_KEY" > /root/.ssh/id_rsa
# chmod 600 /root/.ssh/id_rsa
# ** End of alternative approach

# disable the host key checking.
./deploy/disableHostKeyChecking.sh

# we have already setup the DEPLOYER_SERVER in our gitlab settings which is a
# comma seperated values of ip addresses.
DEPLOY_SERVERS=$DEPLOY_SERVERS

# lets split this string and convert this into array
# In UNIX, we can use this commond to do this
# ${string//substring/replacement}
# our substring is "," and we replace it with nothing.
ALL_SERVERS=(${DEPLOY_SERVERS//,/ })
echo "ALL_SERVERS ${ALL_SERVERS}"

# Lets iterate over this array and ssh into each EC2 instance
# Once inside the server, run updateAndRestart.sh
for server in "${ALL_SERVERS[@]}"
do
  echo "deploying to ${server}"
  ssh ubuntu@${server} 'bash' < ./deploy/updateAndRestart.sh
done
```

### Take the latest changes from GitLab and restart server

This is what we want after we ssh into our box.

* **Delete our own repository and clone again**: Since we are not using any tool like capistrano etc to manage our deployment, a reliable way to do this is to delete the old code and clone again. If you have a better solution than deleting the code, please leave it in comments.
* **npm install**:The catch here is when we do non-interactive ssh into an instance. This means that `~/.bashrc` is not loaded which in turn load our `nvm.sh` file which loads our node. So lets load it explicitly. Here is a great article on [Configuring your login sessions with dot files](http://mywiki.wooledge.org/DotFiles)
* **restart server**: Next problem we need to tackle is pm2 restart. We cannot run pm2 from the source folder as we are deleting this folder and cloning again. PM2 refer the old folder which is not existent and will give error. This is the reason we do not start pm2 daemon from the source repo.

```
#!/bin/bash

# any future command that fails will exit the script
set -e

# Delete the old repo
rm -rf /home/ubuntu/ci_cd_demo/

# clone the repo again
git clone https://gitlab.com/abhinavdhasmana/ci_cd_demo.git

#source the nvm file. In an non
#If you are not using nvm, add the actual path like
# PATH=/home/ubuntu/node/bin:$PATH
source /home/ubuntu/.nvm/nvm.sh

# stop the previous pm2
pm2 kill
npm remove pm2 -g


#pm2 needs to be installed globally as we would be deleting the repo folder.
# this needs to be done only once as a setup script.
npm install pm2 -g
# starting pm2 daemon
pm2 status

cd /home/ubuntu/ci_cd_demo

#install npm packages
echo "Running npm install"
npm install

#Restart the node server
npm start
```

That’s it!! Now every time a commit/merge is made into the master branch, it would be deployed to all the servers.

{% embed url="<https://gitlab.com/tkssharma/ci_cd_demo>" %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://tkssharma-devops.gitbook.io/devops-training/devops-01-continuous-integration/continuous-integration-and-continuous-delivery/ci-cd-using-gitlab-ci-s3/ci-integration-with-aws/ci-integration-with-ec2.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.