How to integrate GitHub repository with Jenkins?
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
Today we will discuss on how to integrate a GitHub repository with Jenkins using GitHub webhook. We will also learn on how to integrate a private GitHub repository with Jenkins and trigger a Jenkins pipeline manually using API.
We will integrate our GitHub repository with Jenkins in two ways
- Publicly accessible Jenkins controller endpoint + GitHub public/private repository (to store the source code) + Automatic trigger (using GitHub Webhook to trigger the pipeline on every push)
- Private Jenkins controller endpoint + GitHub public/private repository (to store the source code) + Manual Trigger (using Scheduled/API trigger to trigger the pipeline)
How to integrate publicly accessible Jenkins controller endpoint with public or private GitHub repository using GitHub webhook?
Warning: This is for demo only, if your Jenkins controller endpoint is exposed to internet then make sure you are using TSL/SSL encryption (HTTPS) for your Jenkins controller before login to Jenkins dashboard and continue with below setup. Or use the 2nd option with private Jenkins endpoint.
Prerequisite
- One EC2 instance with public IP address
- One GitHub private/public repository
Step 1: Install and setup Jenkins on EC2 using Docker.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
## Create an EC2 instance with Amazon Linux 2 AMI and public IP address ## EC2 security group has port 22, 8080 open ## Connect to your EC2 instance ## Install Docker. sudo yum install docker ## Add the ec2-user to the docker group so you can execute Docker commands without using sudo. ## Exit the terminal and re-login to make the change effective sudo usermod -a -G docker ec2-user exit ## Enable docker service sudo systemctl enable docker ## Start docker service sudo systemctl start docker ## Check the Docker service. sudo systemctl status docker ## Run docker jenkins blueocean container docker run \ -u root \ --rm \ -d \ -p 8080:8080 \ -p 50000:50000 \ --name myjenkin \ -v jenkins-data:/var/jenkins_home \ -v /var/run/docker.sock:/var/run/docker.sock \ jenkinsci/blueocean ## get the administrator password docker exec -it myjenkin bash cat /var/jenkins_home/secrets/initialAdminPassword exit ## [ec2-user@ip-172-31-41-155 ~]$ docker exec -it myjenkin bash ## cat /var/jenkins_home/secrets/initialAdminPassword ## exitbash-5.1# cat /var/jenkins_home/secrets/initialAdminPassword ## 67253hgdyu723uyuydf76532vdhe62 => this is your secret ## bash-5.1# exit ## Get your EC2 instance public IP address curl http://169.254.169.254/latest/meta-data/public-ipv4 ## returns your Jenkins controller (EC2) Public IP address ## Open your browser and navigate to Jenikins dashboard and complete the setup ## Provide the initialAdminPassword and completed the setup with suggested plugins ## http://<EC2_Public_IP_Address>:8080/ |
Step 2: Login to your GitHub account and create a public repository with README file (for initial commit) and copy the HTTPS clone URL of your GitHub project.
Step 3: Setup a webhook for Jenkins in your GitHub repository.
Navigate to your GitHub repository => Settings => Webhooks => “Add webhook”
In the “Payload URL”, provide the Jenkins controller IP address followed by github-webhook in “http://<EC2_Public_IP>:8080/github-webhook/
In the “Content type”, select “application/json” and select push event for the webhook trigger. Finally click “Add webhook” to add the Jenkins webhook in GitHub.
Make sure your GitHub webhook for Jenkins is configured properly. The webhook should get a status OK (200) response from your Jenkins controller endpoint.
Step 4: Setup your Jenkins pipeline with GitHub.
Login to your Jenkins dashboard and click “New Item”.
Provide a name to the pipeline and select “Pipeline”. Click “OK”.
Select “GitHub hook trigger for GITScm pooling” and “Build Triggers”. Optionally you can also provide a description for your pipeline.
In the “Pipeline” section, select “Pipeline script from SCM” for “Definition”, “Git” for “SCM” and your GitHub HTTPS/SSH clone URL (for public repository) or SSH clone URL (for private repository).
Note: In case of private repository, you also need to add SSH key in Jenkins credentials and GitHub. This is covered in the Private Jenkins controller endpoint + Private GitHub repository + Scheduled/Manual API trigger section below.
Next, select your GitHub branch name for which you want to trigger your Jenkins pipeline and provide the Jenkinsfile (pipeline definition) location and click “Save”.
Step 4: Create a Jenkinsfile in root for your GitHub repository and push the changes.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
## Clone your GitHub repository git clone <GitHub_Repo_Clone_URL> cd <GitHub_Repo_Name> ## Create a Jenkinsfile ## Replace the label as per your Node label cat <<EOF > Jenkinsfile pipeline { agent { label 'pipeline_demo' } stages { stage('Build') { steps { echo 'building your app!' } } stage('Test') { steps { echo 'testing your app!' } } stage('Deploy') { steps { echo 'deploying your app!' } } } } EOF ## Push the changes to your GitHub repository git add Jenkinsfile git commit -m "pipeline def added" git push |
Note: Replace the label as per your Jenkins label.
The pipeline was successfully executed by GitHub webhook.
How to integrate private Jenkins controller endpoint with a private GitHub repository and trigger the pipeline using scheduled/manual using API?
Prerequisite
- One Jenkins controller
- One private GitHub repository
Step 1: Generate a new SSH key for GitHub.
Step 2: Add the new SSH public key in GitHub.
Step 3: Add the new SSH private key in Jenkins.
Login to your Jenkins dashboard, “Manage Jenkins” => “Manage Credentials” => Select “global” as credential domain => “Add credentials”.
Step 4: Add Jenkinsfile definition to your private GitHub repository.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
## Clone your private GitHub repository using SSH git clone <GitHub_Repo_Clone_SSH_URL> cd <GitHub_Repo_Name> cat <<EOF > Jenkinsfile pipeline { agent { label 'test' } stages { stage('Build') { steps { echo 'building your app!' } } stage('Test') { steps { echo 'testing your app!' } } stage('Deploy') { steps { echo 'deploying your app!' } } } } EOF git add Jenkinsfile git commit -m "pipeline def added" git push |
Note: Replace the label as per your Jenkins node label where you want to execute this pipeline.
Step 5: Setup your Jenkins pipeline with GitHub.
Login to your Jenkins dashboard and click “New Item”.
Provide a name to the pipeline and select “Pipeline”. Click “OK”.
Configure your pipeline settings.
In the “Pipeline” section, select “Pipeline script from SCM” for “Definition”, “Git” for “SCM” and your GitHub SSH clone URL as “Repository URL”, in “Credentials” add the credential created in step 3.
Finally select the branch name and Jenkinsfile location and click “Save”.
Next, we will configure trigger for the pipeline using API or scheduled.
Option 1: Trigger a Jenkins pipeline manually using API:
Step 1: Create an API Token for your Jenkins user.
Step 2: Update the pipeline trigger configuration.
Select “Trigger builds remotely (e.g. – from scripts)” and provide the API token name. Click “Save”.
Step 3: Trigger Jenkins pipeline using API.
|
1 2 3 4 5 6 7 8 9 10 11 |
## Login to shell ## Create some session variables, replace the value accordingly JENKINS_USER=debjeet JENKINS_USER_API_TOKEN=113d06d429808b77d876aac6a8d8712f5b JENKINS_ADDRESS='192.168.0.10:8080' JENKINS_PIPELINE=jenkins_love_github API_TOKEN_NAME=mytoken BASIC_AUTH=${JENKINS_USER}:$JENKINS_USER_API_TOKEN ## Execute the Jenkins pipeline using API curl http://${BASIC_AUTH}@${JENKINS_ADDRESS}/job/${JENKINS_PIPELINE}/build?token=${API_TOKEN_NAME} |
Our Jenkins pipeline executed successfully.
Option 2: Trigger a Jenkins pipeline based on schedule:
Update your pipeline “Build trigger” configuration and select “Build periodically” and provide a cron expression for the schedule. Finally click “Save”
Note: For this demo I have setup the build job to trigger every one minute.
Our pipeline executed as per cron schedule.
Hope you have enjoyed this article, to get more details on Jenkins, please refer below Jenkins official documentation.