Gitlab-CI
gitlab CI another Powerful CI tool
To enable CI/CD Gitlab requires us to follow two steps. They are
    1.
    Add .gitlab-ci.yml to project’s root folder.
    2.
    Setup a Runner.
Add .gitlab-ci.yml:-
In the root folder of your Gitlab project add a file with the name .gitlab-ci.yml . Gitlab uses this file for CI/CD. All your configurations for CI/CD should be added here. Since it is under git project is also part of the version control system. As you change the configurations in this file and push it to Gitlab, It automatically runs the jobs as per your configurations using your Runner configuration.
Setup a Runner:-
In GitLab, Runners run the jobs that you define in .gitlab-ci.yml. A Runner can be a virtual machine, a VPS, a bare-metal machine, a docker container or even a cluster of containers. Here I use the default Gitlab runner. If you want to configure your own runner Please check gitlab ci/cd documentation.
Now let's try different configurations.
    1.
    A Basic Job:-
Let’s start in a traditional way. I mean let us create a basic job that says “Hello world” 😃
The syntax for declaring a job in .gitlab-ci.yml is as follows.
1
job name:
2
script:
3
- your shell script.
4
- more script.....
5
Ex:-
6
sayhello:
7
script:
8
- echo "Hello world"
Copied!
Tips:-
    .gitlab-ci.yml should contain at least one job. If there are no jobs defined in it when you push the code to Gitlab, Gitlab runner will throw an error saying invalid yaml configuration. the job running fails.
    Make sure you validate .gitlab-ci.yml before you commit and Gitlab. If there is any validation error you can get it during the validation process. I use Gitlab Workflow ❣️, A great plugin for git operations with visual studio code.
    You can see all the pipelines of a project under the Pipelines section of Gitlab repository.
    You can see the logs of the job by clicking on it. ✅ button indicates your job ran successfully. Yellow indicates your job is paused. ❌ indicates that your job failed. Blue represents that the job is running.
    Any .gitlab-ci.yml supported keywords can't be used as a job name.
Once you commit the above code and push it to Gitlab, it should run a job with the specified name and execute the script you mentioned under script tag.gitlab pipelines section which shows all the jobs running and ran with respective status.
2. Multiple jobs:-
Okay now we know how to write a single job, Can we write multiple jobs in .gitlab-ci.yml?
The answer is Yes. You can write as many as jobs you want. The Only condition is that Jobs should have different names. Let’s write 2 jobs in which one says hi and the other says Bye.
1
#A job with sayHello name
2
sayHello:
3
script:
4
- echo "hi"
5
#A job with sayBye name
6
sayBye:
7
script:
8
- echo "Bye"
Copied!
Tips:-
    Make sure you don’t have multiple jobs with the same name If your config has multiple jobs with the same name, beginning jobs will be skipped and the last job will be executed.
    The default stage for a given job is test.
The output should look like this if you click on the pipeline id which starts with #.Multiple job configurations.
3. Jobs with stages:-
Generally, every project will have stages before deploying like clean, build, test and deploy. Each stage will have a set of commands to execute. How do we configure jobs in with stages in .gitlab-ci.yml ?
1
#stages can be declared using a keyword stages. You can add any number of stages as you require.
Copied!
1
stages:
2
- clean
3
- build
4
- test
5
- deploy
6
#Lets define a job with for every stage.
7
jobCleaning:
8
stage: clean # This tag tells gitlab to run this job only for clean stage
9
script:
10
- echo "Cleaning the code"
11
jobBuilding:
12
stage: build # This tag tells gitlab to run this job only for build stage
13
script:
14
- echo "Building the code"
15
jobTesting:
16
stage: test # This tag tells gitlab to run this job only for test stage
17
script:
18
- echo "Testing the code"
19
jobDeploying:
20
stage: deploy # This tag tells gitlab to run this job only for deploy stage
21
script:
22
- echo "Deploying the code"
23
Copied!
image showing stages, respective job configurations.
Tips:-
    Every stage need not have a job declared.
    Order of the job execution is based on the order of stage declaration. If a stage doesn’t have at least one job defined, Then that stage will be skipped and goes to next stage job.
    A stage can have multiple jobs defined.
Image showing multiple jobs under a stage.
    if you have defined a stage called “test”, and define a job without stage added in it, Then the job will run under test stage since the default stage is test.
    If you haven’t defined the test stage and create a job without stage defined in it, Then Gitlab pipeline will throw an error like yaml invalidand will ask you to specify the stage for the job as shown below.
pipeline failure caused by not specifying stage parameter to the job named commonJob.
4. Branch-specific jobs:-
By default with the previous configurations we had, Whenever you make a push to any other branch the pipeline will get triggered on the branch. This setup is good if you have common functionality to implement. But let’s try to add a config which runs only when you push the code to dev branch.
This can be done very simply by adding only tag to a job.
1
stages:
2
- clean
3
- build
4
- test
5
- deploy
Copied!
1
jobCleaning:
2
stage: clean
3
script:
4
- echo "Cleaning the code"
5
only:
6
-dev
7
jobBuilding:
8
stage: build
9
script:
10
- echo "Building the code"
11
only:
12
-dev
13
jobTesting:
14
stage: test
15
script:
16
- echo "Testing the code"
17
only:
18
-dev
19
jobDeploying:
20
stage: deploy
21
script:
22
- echo "Deploying the code"
23
only:
24
-dev
Copied!
As discussed above snippet will run on all operation you perform on dev branch i.e for merge requests, issues, pushes.
Tips:-
    only will accept regular expressions. If your commit matches with regular expression, pipeline gets activated and jobs will executed as per configuration.
    allowed values to only tag are as follows.
allowed values to
But what if you want to run your job only for issue commits or new merge requests only?
5. Excepting a job from pipeline:-
What if your requirement is not to run your pipeline for a certain case?
We can achieve this by providing exceptto the job configuration. See the following example
1
stages:
2
- clean
3
- build
4
- test
5
- deploy
6
jobCleaning:
7
stage: clean clean stage
8
script:
9
- echo "Cleaning the code"
10
only:
11
- dev
12
except:
13
- merge_requests # this job will not run for all merge requests on dev branch
14
jobBuilding:
15
stage: build
16
script:
17
- echo "Building the code"
18
only:
19
- dev
20
jobTesting:
21
stage: test
22
script:
23
- echo "Testing the code"
24
only:
25
-dev
26
jobDeploying:
27
stage: deploy
28
script:
29
- echo "Deploying the code"
30
only:
31
-dev
Copied!
If your job has both only, except configurations, The flow of execution as follows
    only and except are inclusive. If both only and except are defined in a job specification, the ref is filtered by only and except.
    only and except allow the use of regular expressions (using Ruby regexp syntax).
    only and except allow to specify a repository path to filter jobs for forks.
6. Fail-safe jobs:-
By default If a pipeline contains multiple jobs in multiple stages, If an error occurs at any job, Next jobs will be skipped with a pipeline status as failed.Image showing job failure and skipping remaining jobs in the pipeline.
But If we still want to proceed and continue the pipeline, we can achieve this by adding a config param called allow_failure. It accepts either true or false . The default value is false. That is the reason if any error occurs in pipeline remaining jobs will be skipped.
See the below example configuration.
1
stages:
2
- clean
3
- build
4
- test
5
- deploy
6
jobCleaning:
7
stage: clean clean stage
8
allow_failure: true #this tells gitlab to not skip other jobs in the pipeline if any error occurs in the current job.
9
script:
10
- echo "Cleaning the code"
11
only:
12
- dev
13
except:
14
- merge_requests # this job will not run for all merge requests on dev branch
15
jobBuilding:
16
stage: build
17
script:
18
- echo "Building the code"
19
only:
20
- dev
21
jobTesting:
22
stage: test
23
script:
24
- echo "Testing the code"
25
only:
26
-dev
27
jobDeploying:
28
stage: deploy
29
script:
30
- echo "Deploying the code"
31
only:
32
-dev
Copied!
Image showing job failure but still continuing remaining jobs in the pipeline.
Tips:-
    If you want to retry the job in case of failure again for a certain number of times can be configured using retry in the job configuration.
    If retry is set to 2, and a job succeeds in a second run (first retry), it won’t be retried again.
    retry value has to be a positive integer, equal to or larger than 0, but lower or equal to 2 (two retries maximum, three runs in total).
7. Running Jobs at a certain job status:-
Okay, We have tried faile safe jobs and retrying them too. But What if we need to implement a job which runs if a job fails or a job runs successfully or even always despite the job status?
This can be achieved by using the when tag of the job configuration.
Allows values for the when tag are
    on_success - execute job only when all jobs from prior stages succeed (or are considered succeeding because they are marked allow_failure). This is the default.
    on_failure - execute job only when at least one job from prior stages fails.
    always - execute job regardless of the status of jobs from prior stages.
    manual - execute job manually (added in GitLab 8.10). Read about manual actions below.
Let's see an example which runs a job called jobSuccess for the successful running of the job, jobFailure which runs on job failure, runAlways which runs always.
1
stages:
2
- clean
3
- build
4
- test
5
- deploy
6
jobCleaning:
7
stage: clean clean stage
8
script:
9
- echo "Cleaning the code"
10
only:
11
- dev
12
except:
13
- merge_requests # this job will not run for all merge requests on dev branch
14
jobBuilding:
15
stage: build
16
script:
17
- echo "Building the code"
18
only:
19
- dev
20
jobTesting:
21
stage: test
22
script:
23
- echo "Testing the code"
24
only:
25
-dev
26
jobDeploying:
27
stage: deploy
28
script:
29
- echo "Deploying the code"
30
only:
31
-dev
32
jobSuccess:
33
script:
34
- echo "Running on successful running of the job."
35
when: on_success
36
stage: test
37
jobFailure:
38
script:
39
- echo "Running on failure of the job."
40
when: on_failure
41
stage: test
42
jobAlways:
43
script:
44
- echo "Running always job"
45
when: always
46
stage: test
Copied!
Image showing pipeline which has jobSuccess, jobFailure, jobAlways which are configured with when field.
Note:-
    Observe that jobFailure is skipped because jobTesting is not Failed. It will run if jobTestingfails.
8. Defining variables in jobs:-
Gitlab allows you to create variables in the gitlab-ci.yml .These variables can be later used in the job configurations. A variable declaration should followkey:value format.
There are two types of variables in Gitlab CI/CD.
    Global variables.
    Job specific variables.
Both types of variables can be defined by using a tag called variables .
If you define variables in Job configuration, variables under it become job specific variables and will override the global variables with the same name are defined.
If you define variables as root tag, Values under it will become global variables. These can be accessed in any job under the pipeline.
Once you define the variables, They can be accessed using
1
# Global variables. These variables can be accessed in any job.
2
variables:
3
BRANCH_NAME: "dev"
4
CLEAN_COMMAND: "mvn clean"
5
OVERRIDE_VARIABLE: "OVERIDE BEFORE"
6
stages:
7
- clean
8
- build
9
- test
10
- deploy
11
jobCleaning:
12
stage: clean clean stage
13
variables: #Job specific variables.
14
SECRET_KEY: "This is secret key"
15
OVERRIDE_VARIABLE: "OVERIDED AFTER" #Varibale name is same as gloabl variable. SO this variable should be overriden.
16
retry: 1
17
script:
18
- echo "Cleaning the code"
19
- echo "${OVERRIDE_VARIABLE}" # should print OVERIDED AFTER.
20
- echo "${CLEAN_COMMAND}"
21
only:
22
- dev
23
except:
24
- merge_requests # this job will not run for all merge requests on dev branch
25
jobBuilding:
26
stage: build
27
script:
28
- echo "Building the code"
29
only:
30
- dev
31
jobTesting:
32
stage: test
33
script:
34
- echo "Testing the code"
35
only:
36
-dev
37
jobDeploying:
38
stage: deploy
39
script:
40
- echo "Deploying the code"
41
only:
42
-dev
43
Copied!
Job log which shows job specific and global variables.
These are some of the cases I have tried. There are more advanced topics like cache, artifacts, environments, include etc. I will write about them in another blog. If you want to read more about Gitlab CI/CD refer below document.
Last modified 2yr ago
Copy link