Setup Jenkins to run as a different user and automate the Deployment with Git (GitHub/ GitLab/ BitBucket) for Magento 1 and Magento 2

February 14, 2023

Intended Audience: System Administrators with primary knowledge, Developers with good experience in hosting web applications.

Before learning the syntax, let’s just spend a few minutes to understand the purpose of installing Jenkins in the first place.

Why CI & CD?

Businesses are growing dynamically and rapidly today more than ever. Softwares act as a crucial part of running business functions smoothly, and sometimes software is the main revenue generation point of a business. Therefore it is essential to deploy any bug or a feature to production soon after the software development team develops it.

Another use case of CI/CD is to minimise the conflicts between developers’ code by adding them to develop branches continuously. Additionally running test cases when merging branches will also generate the errors in the early stages of the deployment process.

What is CI & CD?

CI and CD is a common abbreviation in the IT industry for Continuous Integration and Continuous Delivery or Continuous Deployment.

Image is originally from https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment

Continuous Integration

Continuous integration is when the developer merges their branches to the main branch as soon as they finish the developer testing on partial implementation, completion of a feature or a bug. Upon merging unit tests will run to make sure that the functionality is running according to the test cases.

Continuous Delivery

Continuous Delivery means any code passes the test cases when margin to the staging branch will automatically deploy to a staging environment. If you implement only continuous delivery, probably after manual testing, you can trigger the production deployment. If you are planning agile scrum, this would be ideal as you do not want to worry about the breaking of the Production Environment in the middle of the sprint.

Continuous Deployment

Continuous Deployment is the next level of Continuous Delivery. In Continuous Delivery, we deployed to production manually after the testing in staging. However, with Continuous Deployment, code will automatically merge through branches and deploy to the Production Environment upon merging it to the main branch upon successful development testing. The only thing that stops developers merge getting deployed to production will be failing a test case.

Both of the CD methods have their pros and cons. However, they are not in the scope of this document to discuss. I would recommend reading through the Atlassian article in reference for more detail.

Scope of this article:
This article only intended to explain setting up Jenkins server, Running it under a different user than Jenkins (we will run it with deploy user).

Additional Read:

To see setup AWS EC instances with LEMP stack that run with deploy users, please refer to this[Link] article.

Server Stack

LEMP – Linux Nginx MySql PHP-FPM
Type: Virtual Dedicated Server
Linux Flavor: Ubuntu 18.04 LTS
User: Non-root user with sudo access
Run as User: deploy

Install Java for Jenkins

Install the latest JRE

More clarification refers to the Digital Ocean article on How To Install Java with Apt on Ubuntu 18.04 referenced in the link.

Update the repositories
$ sudo apt update

Check whether java is already installed
$ java -version
If you don’t get something like below, you need to install Java Runtime Environment (JRE) to your server.

Install JRE with below command.
$ sudo apt install default-jre

When you run the java -version command, you should get a result like above.

Install latest JDK

You also need to install the Java Development Kit (JDK) for installing Jenkins successfully.

Run below command to install JDK.
$ sudo apt install default-jdk

Setting up JAVA_HOME

Setting the JAVA_HOME Environment Variable will make sure the Java-related application runs smoothly.

Check whether $JAVA_HOME is already configured.
$ echo $JAVA_HOME

If not check the java alternative list for java
$ update-java-alternatives –list

Run the following command to setup JAVA_HOME (According to the above alternatives)
$ export JAVA_HOME=&ltpath_to_java&gt
Eg:
$ export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64

Now if you echo $JAVA_HOME, it should display as below.

Installing Jenkins

First, add the repository key related to the source list.
$ wget -q -O – https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add –

Now let’s add the source to the source.list
$ sudo sh -c ‘echo deb http://pkg.jenkins-ci.org/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list’

Run the update so apt can use the new repository.
$ sudo apt update

Finally, it’s time to install the Jenkins
$ sudo apt install jenkins

Optional:
However, if you are setting up Jenkins for production, it is highly recommended to use an SSL certificate for the domain to protect passwords. Please refer below article to set up Jenkins with SSL.
https://www.digitalocean.com/community/tutorials/how-to-configure-jenkins-with-ssl-using-an-nginx-reverse-proxy-on-ubuntu-18-04

Configuring Jenkins to run with a different users (Same user as Nginx)

Check the status of the Jenkins with the following command.
$ sudo service jenkins status
If the installation is successful, you should see something like below.

Run the following command to see which user runs Jenkins.
$ ps aux | grep jenkins

Check the Nginx user
$ ps aux | grep nginx

As Jenkins and the Nginx are running with different users, there can be conflicts when doing the deployments. To avoid this best approach is to change the Jenkins user to www-data (Or to change the Nginx users).

Steps to change the Jenkins user.

Stop Jenkins
$ sudo service jenkins stop

Check whether Jenkins is running any more.
$ ps aux | grep jenkins

If there are no Jenkins processes, you can proceed with the steps.
$ sudo vi /etc/default/jenkins

Change the JENKINS_USER parameter value as below. (You can change the group value as well)

Also change the owner of the following files and folders.
$ sudo chown -R www-data:jenkins /var/lib/jenkins
$ sudo chown -R www-data:jenkins /var/log/jenkins
$ sudo chown -R www-data:jenkins /var/cache/jenkins

Restart Jenkins
$ sudo service jenkins restart

Check whether the new user is running and the Jenkins user
$ sudo service jenkins status

You also need to check whether the process is running accurately.
$ ps aux | grep jenkins

Configuring jenkins

Once Jenkins installation is finished, you can access the installation by using the below link.

&ltserver_ip&gt:8080
E.g. http://10.0.1.2:8080

You have to use the password in a given location to log in as an administrator.
$ cat /var/lib/jenkins/secrets/initialAdminPassword
Select suggested plugins

This process will take some time to complete.

Enter the credentials and create the first admin user here.

Next, Confirm the URL – Can keep the default.
Proceed to confirm using Jenkins.

Setup Deployment

Create a New Job

Let’s create a cicd folder in the Nginx running users’ home to clone the code in the process.

Eg: /home/www-data/cicd
If the home/&ltuser&gt folder does not exist, create it and make the Nginx running user as the owner.
$ mkdir /home/www-data

Create the following sub folders and give permission as shown below.
$ mkdir /home/www-data/cicd/source/&ltproject-name&gt
$ mkdir /home/www-data/cicd/config/&ltproject-name&gt
$ chown -Rf www-data:www-data /home/www-data/
Click on the “New Item” link.
Enter item name & Click on Freestyle Project.
Click on the Advanced button on the “General” tab.
Tick “Use custom workspace” and Enter the folder to download the git repository.
/home/www-data/cicd/source
Scroll down to the source code management section and select Git.
Get the git clone URL from your repository (https) and enter it as below.
Add the credentials for Git by clicking on “Add” -> Jenkins.
Fill the following popup.
Mention the correct branch here.
Scroll down to the Build section and click on “Add build Setup”.
In the dropdown select “Invoke Ant”.
Give the build file name as follows (Default will be build.xml)
Save the settings.
Install Ant in the server.
$ sudo apt install ant

One more step to set up to conclude the setting up process; That is to place the build.xml file for Ant in the document root.
In our case it’s build_staging.xml as we have overridden the default build.xml to build_staging.xml in the “Build File” field above.

Create build.xml

build.xml is a way to instruct Jenkins to execute a series of commands for the build process. The syntax is self-explanatory, and you can change it as you wish. Here, I am only focussing on automating M1, and you can always change the setting per your requirement.

Eg: Magento 1 example

				
					build.xml
<?xml version="1.0" encoding="UTF-8"?>
<project name="Nomin M1 Staging" default="run task">
   <property name="basedir" value="."/>
   <target name="sync-files" description="Sync files">
       <exec executable="bash" failonerror="true" newenvironment="false">
           <arg line="rsync_staging.sh"/>
       </exec>
   </target>
   <target name="cache-flush" description="Delet default and amasty fpc cache folders">
       <exec executable="rm" failonerror="true">
           <arg line="-rf /var/www/html/eshop-test-git/var/cache /var/www/html/eshop-test-git/var/amasty_fpc/ "/>
       </exec>
   </target>
   <target name="reindex" description="Reindex all the indexes">
       <exec executable="php" failonerror="true">
           <arg line="/var/www/html/eshop-test-git/shell/indexer.php reindexall"/>
       </exec>
   </target>
   <target name="run task" description="Run target one by one">
       <sequential>
           <antcall target="sync-files"/>
           <antcall target="cache-flush"/>

       </sequential>
   </target>
</project>

rsync_staing.sh

#!/usr/bin/env bash
rsync -avv –exclude=app/etc/local.xml ./app/ /var/www/html/eshop-test-git/app/ –delete
rsync -avv ./errors/ /var/www/html/eshop-test-git/errors/ –delete
rsync -avv ./includes/ /var/www/html/eshop-test-git/includes/ –delete
rsync -avv ./js/ /var/www/html/eshop-test-git/js/ –delete
rsync -avv ./lib/ /var/www/html/eshop-test-git/lib/ –delete
rsync -avv ./other/ /var/www/html/eshop-test-git/other/ –delete
rsync -avv ./shell/ /var/www/html/eshop-test-git/shell/ –delete
rsync -avv ./skin/ /var/www/html/eshop-test-git/skin/–delete
rsync -avv ./.gitignore /var/www/html/eshop-test-git/.gitignore
cp -fv ../../config/nomin-m1/local.xml /var/www/html/eshop-test-git/app/etc/

Eg: Magento 2 example

				
					<?xml version="1.0" encoding="UTF-8"?>
<project name="Nomin M1 Staging" default="run task">
   <property name="basedir" value="."/>

   <target name="sync-app" description="Sync app folder">
       <exec executable="rsync" failonerror="true">
           <arg line="-rv app/ /<document_root_absalute_path>/app/"/>
       </exec>
   </target>
   <target name="sync-skin" description="Sync skin folder">
       <exec executable="rsync" failonerror="true">
           <arg line="-rv skin/ /<document_root_absalute_path>/skin/"/>
       </exec>
   </target>
   <target name="sync-local-xml" description="Copy Local XML">
       <exec executable="rsync" failonerror="true">
           <arg line="-rv ../../config/nomin-m1/local.xml /<document_root_absalute_path>/app/etc/local.xml"/>
       </exec>
   </target>
   <target name="cache-flush" description="Delete default and amasty fpc cache folders">
       <exec executable="rm" failonerror="true">
           <arg line="-rf /<document_root_absalute_path>/var/cache /<document_root_absalute_path>/var/amasty_fpc/ "/>
       </exec>
   </target>
   <target name="reindex" description="Reindex all the indexes">
       <exec executable="php" failonerror="true">
           <arg line="/<document_root_absalute_path>/shell/indexer.php reindexall"/>
       </exec>
   </target>
   <target name="run task" description="Run target one by one">
       <sequential>
           <antcall target="sync-app"/>
           <antcall target="sync-skin"/>
           <antcall target="sync-local-xml"/>
           <antcall target="cache-flush"/>
           <antcall target="reindex"/>
       </sequential>
   </target>
</project>

Eg: Magento 2

				
					<?xml version="1.0" encoding="UTF-8"?>
<project name="php testing" default="run task">
   <property name="basedir" value="."/>

   <target name="sync" description="Sync New code">
       <exec executable="rsync" failonerror="true">
           <arg line="-rv app/ /<document_root_absalute_path>app/"/>
       </exec>
   </target>
   <target name="copy-composer" description="copy extension to test server">
       <exec executable="cp" failonerror="true">
           <arg line="-nf composer.* /<document_root_absalute_path>"/>
       </exec>
   </target>
   <target name="composer-install" description="Install composer updates">
       <exec executable="composer" failonerror="true">
           <arg line="install -d /<document_root_absalute_path>"/>
       </exec>
   </target>
   <target name="deploy-mode-set" description="set up and upgrade">
       <exec executable="php" failonerror="true">
           <arg line="/<document_root_absalute_path>bin/magento deploy:mode:set production"/>
       </exec>

   </target>
   <target name="run task" description="Run target one by one">
       <sequential>
           <antcall target="sync"/>
           <antcall target="copy-composer"/>
           <antcall target="composer-install"/>
           <antcall target="deploy-mode-set"/>
       </sequential>
   </target>
</project>

Conclusion

I have done the basic automation here. With adding better instructions and sometimes using a bash script, you will be able to do a significantly optimised CI/CD process.

Ref:
https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment
https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-on-ubuntu-18-04
https://www.digitalocean.com/community/tutorials/how-to-install-jenkins-on-ubuntu-18-04

ABOUT THE AUTHOR

Muditha Ediriweera is passionate about Magento and has nearly a decade of experience in Magento eCommerce development. He has developed a lot of Magento 1 and Magento 2 sites with both OpenSource (CE) and Commerce (EE) platforms. His experience is not limited as a Developer as he guides a lot of people to discover Magento platform capabilities in both code level as a Magento Trainer and features wise as a Magento solution consultant. Muditha is currently helping merchant to sell products online with Magento as a Founder & CEO at NeoSolax with his high calibre Magento development team.

Tags :

Beginner,Magento 2,Training