.gitlab-ci.yml Part Five – Basics Of Environment Need Retry Timeout Parallel
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed .gitlab-ci.yml features like tags, allow_failure and when.
https://cloudaffaire.com/gitlab-ci-yml-part-four-basics-of-tags-allow-failure-when/
In this blog post, we will discuss .gitlab-ci.yml features like environment, need, retry, timeout and parallel.
environment:
Environments allow control of the continuous deployment of your software, all within GitLab. Environments are like tags for your CI jobs, describing where code gets deployed. Environment is used to define a job deploys to a specific environment. If the environment is specified and no environment under that name exists, a new one will be created automatically. The environment supports the below parameters:
- name: The name of the environment. like prod, dev, uat etc. Common names are qa, staging, and production, but you can use whatever name works with your workflow. Instead of defining the name of the environment right after the environment keyword, it is also possible to define it as a separate value. For that, use the name keyword under environment.
- url: This is an optional value that when set, it exposes buttons in various places in GitLab which when clicked take you to the defined URL.
- on_stop: Closing (stopping) environments can be achieved with the on_stop keyword defined under environment. It declares a different job that runs in order to close the environment.
- action: The action keyword is to be used in conjunction with on_stop and is defined in the job that is called to close the environment.
- auto_stop_in: The auto_stop_in keyword is for specifying life period of the environment, that when expired, GitLab GitLab automatically stops them.
- Kubernetes: The kubernetes block is used to configure deployments to a Kubernetes cluster that is associated with your project.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
################################################################################ ## .gitlab-ci.yml Part Five: environment | need | retry | timeout | parallel ## ################################################################################ ## Prerequisites: GitLab Access Token ## https://cloudaffaire.com/how-to-create-a-gitlab-project-using-api/ ## One CentOs system with internet access ## git, curl and jq package installed ## ------------------------------------ ## Create A GitLab Project To Play With ## ------------------------------------ ## Create a project in GitLab ## GitLab Console => Projects => Your Projects => New Project => PROJECT_ID=$(curl --silent --header "PRIVATE-TOKEN: -XPOST "https://gitlab.com/api/v4/projects?name=cloudaffairecicd&visibility=private&initialize_with_readme=true" | jq '.id') ## Get the ssh clone url for your project ## GitLab Console => Projects => Your Projects => GIT_SSH_CLONE_URL=$(curl --silent --header "PRIVATE-TOKEN: -XGET "https://gitlab.com/api/v4/projects/$PROJECT_ID" | jq -r '.ssh_url_to_repo') ## Clone your GitLab project repository git clone $GIT_SSH_CLONE_URL ## ----------- ## environment ## ----------- ## Create the pipeline configuration file cd cloudaffairecicd && vi .gitlab-ci.yml --------------------- mygitlabjob_one: stage: deploy script: echo "this is production" environment: name: production url: https://cloudaffaire.com --------------------- :wq ## Push the .gitlab-ci.yml file into GitLab git add . git commit -m "job with environment" git push |
needs:
The needs: keyword enables executing jobs out-of-order, allowing you to implement a directed acyclic graph in your .gitlab-ci.yml. This lets you run some jobs without waiting for other ones, disregarding stage ordering so you can have multiple stages running concurrently.
needs Requirements and limitations:
- If needs: is set to point to a job that is not instantiated because of only/except rules or otherwise does not exist, the pipeline will be created with YAML error.
- There is a limit, ranges between 10 to 50 on the maximum number of jobs that a single job can need in the needs: array:
- If needs: refers to a job that is marked as parallel:. the current job will depend on all parallel jobs created.
- needs: is similar to dependencies: in that it needs to use jobs from prior stages, meaning it is impossible to create circular dependencies. Depending on jobs in the current stage is not possible either, but support is planned.
- Related to the above, stages must be explicitly defined for all jobs that have the keyword needs: or are referred to by one.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
## ---- ## need ## ---- ## Edit the pipeline configuration file cat /dev/null > .gitlab-ci.yml && vi .gitlab-ci.yml --------------------- one: stage: build script: echo "this will get executed 1st" two: stage: build script: echo "this will get executed 1st" three: stage: test script: echo "this will get executed 2nd" four: stage: test script: echo "this will get executed 2nd" five: stage: test script: echo "this will get executed 2nd" six: stage: deploy script: echo "this will get executed 3rd" --------------------- :wq ## Push the .gitlab-ci.yml file into GitLab git add . git commit -m "job without needs" git push ## Edit the pipeline configuration file cat /dev/null > .gitlab-ci.yml && vi .gitlab-ci.yml --------------------- one: stage: build script: echo "this will get executed 1st" two: stage: build script: echo "this will get executed 1st" three: stage: test needs: [] script: echo "this will get executed 1st" four: stage: test needs: ["one"] script: echo "this will get executed 2nd" five: stage: test needs: ["two"] script: echo "this will get executed 2nd" six: stage: deploy script: echo "this will get executed 3rd" --------------------- :wq ## Push the .gitlab-ci.yml file into GitLab git add . git commit -m "job with needs" git push |
retry:
retry allows you to configure how many times a job is going to be retried in case of a failure. When a job fails and has retry configured, it is going to be processed again up to the amount of times specified by the retry keyword. If retry is set to 2, and a job succeeds in a second run (first retry), it won’t be retried again. retry value has to be a positive integer, equal or larger than 0, but lower or equal to 2 (two retries maximum, three runs in total). By default, a job will be retried on all failure cases. To have a better control on which failures to retry, retry can be a hash with the following keys:
- max: The maximum number of retries.
- when: The failure cases to retry.
Possible values for when are:
- always: Retry on any failure (default).
- unknown_failure: Retry when the failure reason is unknown.
- script_failure: Retry when the script failed.
- api_failure: Retry on API failure.
- stuck_or_timeout_failure: Retry when the job got stuck or timed out.
- runner_system_failure: Retry if there was a runner system failure (e.g. setting up the job failed).
- missing_dependency_failure: Retry if a dependency was missing.
- runner_unsupported: Retry if the runner was unsupported.
- stale_schedule: Retry if a delayed job could not be executed.
- job_execution_timeout: Retry if the script exceeded the maximum execution time set for the job.
- archived_failure: Retry if the job is archived and cannot be run.
- unmet_prerequisites: Retry if the job failed to complete prerequisite tasks.
- scheduler_failure: Retry if the scheduler failed to assign the job to a runner.
- data_integrity_failure: Retry if there was a structural integrity problem detected.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
## ----- ## retry ## ----- ## Edit the pipeline configuration file cat /dev/null > .gitlab-ci.yml && vi .gitlab-ci.yml --------------------- mygitlabjob_one: stage: deploy script: echos "this job will fail and you can retry twice" retry: 2 --------------------- :wq ## Push the .gitlab-ci.yml file into GitLab git add . git commit -m "job with retry" git push |
timeout:
Timeout allows you to configure a timeout for a specific job. The job-level timeout can exceed the project-level timeout but can not exceed the Runner-specific timeout.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
## ------- ## timeout ## ------- ## Edit the pipeline configuration file cat /dev/null > .gitlab-ci.yml && vi .gitlab-ci.yml --------------------- mygitlabjob_one: stage: deploy script: - echo "this job will timeout after 1 minute" - sleep 120 timeout: 1m --------------------- :wq ## Push the .gitlab-ci.yml file into GitLab git add . git commit -m "job with timeout" git push |
parallel:
parallel allows you to configure how many instances of a job to run in parallel. This value has to be greater than or equal to two (2) and less than or equal to 50. This creates N instances of the same job that run in parallel. They’re named sequentially from job_name 1/N to job_name N/N. For every job, CI_NODE_INDEX and CI_NODE_TOTAL environment variables are set. Marking a job to be run in parallel requires adding parallel to your configuration file.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
## -------- ## parallel ## -------- ## Edit the pipeline configuration file cat /dev/null > .gitlab-ci.yml && vi .gitlab-ci.yml --------------------- myjob: stage: deploy script: - echo "5 instance of job will be created" parallel: 5 --------------------- :wq ## Push the .gitlab-ci.yml file into GitLab git add . git commit -m "job with parallel" git push ## Delete your project curl --silent --header "PRIVATE-TOKEN: -XDELETE "https://gitlab.com/api/v4/projects/$PROJECT_ID" | jq '.' cd .. && rm -rf cloudaffairecicd |
Hope you enjoyed this article. In the next blog post, we will continue our discussion .gitlab-ci.yml.
To get more details on GitLab you can follow the below link.
https://docs.gitlab.com/ee/README.html
To Get more details on Git you can follow the below links.
https://cloudaffaire.com/category/devops/git/