CI using AWS, Jenkins, Git and PowerShell

During the last two years there have been a lot of discussions about DevOps and winops (DevOps on the Windows stack). Personally, I think it is a little challenging to do DevOps in the Microsoft stack because the tools were not available before but now are becoming available and overtime it will be second nature.

What I  usually tell people is that if  they want to do devops start small, take baby steps and start focusing on how to improve processes, source control and  automation. Why processes ? Because automating a pile of crap does not make it gold, it makes it a pile of automated crap

On this post, I want to show you how to set up a Continuous Integration pipeline using Jenkins to check if your PowerShell adhere with the Script Analyzer default rules. In this simple exercise that I will show you how to use different tools and technologies to do this.

The technologies that will be used in this post is AWS, Red Hat, Jenkins, Windows 2012 r2, Git and SSH. I am going to run Jenkins on Red Hat 7. You can also run this on your Lab there is no need to run it in the cloud

Prerequisites

  1. Create a AWS account. This is free and you can create machines on their free tier
  2. Create  a GitHub account. This is free and will be used to upload your PowerShell code
  3. On your personal computer download and  install GIT https://git-scm.com
  4. If you are using a PC download and install www.putty.org/

 

The Blog is divided on the following steps

AWS INSTANCE CREATION

JENKINS INSTALLATION

JENKINS CONFIGURATION

WINDOWS 2012 R2 BUILD SERVER

GITHUB CONFIGURATION

PROJECT CREATION

CONCLUSION

AWS INSTANCE CREATION

Login to your AWS account -> console.aws.amazon.com

1.png

 

Click on services -> Compute and select EC2

2.png

Click on launch  and then select Red Hat Enterprise Linux 7.2 AMI

3

For instance type choose t2.micro this is the free tier and then click next: Configure Instance Details

4.png

Make sure to select the correct subnets you have configured in your VPC

5.png

 

Click next until you get to the configuration of security groups and make sure you allow all traffic for this lab

6.png

Click on review and lunch and make sure you have access to your private key

7

 

Now you will see now the instance being created with a public IP assigned

8.png

You are going ssh to your instance default ec2-user and your personal key

From Mac ssh -i key.pem ec2-user@IPaddress

To use putty follow this instructions to set it up  http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/putty.html

9.png

Now that you are log in, the first thing to do is to update the OS using the following command

sudo yum update -y

After is done please restart using the reboot command

10.png

11.png

 

JENKINS INSTALLATION

Type the following commands

  1. sudo bash
  2. yum install vim -y
  3. yum install java -y
  4. yum install wget -y
  5. yum install git -y
  6. wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkinsci.org/redhat/jenkins.repo

12.png

7.  rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key

8. yum install jenkins -y

13.png

9. service jenkins start/stop/restart

10. chkconfig jenkins on

11. service jenkins start

Jenkins should be installed on the server, To make sure is working open a browser and type the external IP address using port 8080

15.png

If you have questions with the installation you can look at Jenkins wiki https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Red+Hat+distributions

 

JENKINS CONFIGURATION

The first thing we need to do is unlock Jenkins. run the following command

cat /var/lib/jenkins/secrets/initialAdminPassword

After you get the password paste on the Jenkins webpage

16

For the lab I would install all the suggested plugins

17.png

After the plugins installation create a new account

Now that you are login lets install the PowerShell plugin. Click on manage plugins

18

Click on Available, find PowerShell and all GitHub plugins and then click download now and install after restart. Make sure to restart Jenkins

19.png49.png

WINDOWS 2012 R2 BUILD SERVER

Now we need a build server where script analyzer will be ran from

  1. Go Back to AWS and click on create new instance using the Free Windows 2012 R2 AMI20
  2. Choose the same options as RHEL server. Creating this instance will take like 5 to 10 minutes
  3. To  get the password click on get Windows password after you have selected the Windows EC2 instance21.png
  4. After you have decrypt the password you can login to the server

After you have login to the server lets do the following

  • After you have rebooted the server run the following command “Install-Module -Name PSScriptAnalyzer”
  • Type get-ScriptAnalyzer and you should get sometime like this25.png

Now we need to install Java jre 64 bit version. Make sure to install the 64 bit version because when running a Jenkins job it will need to talk to the 64 bit  version of PowerShell

http://www.java.com/en/download/manual.jsp

26.png

Make sure that Java is installed by opening and new PowerShell window and type “java -version”

27.png

Finally install GIT with the default settings https://git-scm.com/download/win

53.png

Make sure to reboot the computer after done with the installations

CONFIGURE WINDOWS BUILD SERVER ON JENKINS

Click on Jenkins -> Manage Jenkins -> Manage Nodes

28.png

Add node give it a name  and check on permanent agent

30.png

Open Jenkins web console using the internal IPaddress on the Windows server and to make sure you do not get block disable the Windows Firewall

32

When try to install the agent on the server you may see a message regarding slaveagentport.disabled

33.pngTo enable go to Jenkins -> Manage Jenkins -> Configure Global Security and then select random for TCP port for JNLP agents. When doing this in production select a fixed port.

34.png

Now if you go back to manage nodes you should see a launch icon

35

Now click on file and install as service

36

Now you can check the status of the node in Jenkins

37.png

 

GITHUB CONFIGURATION

Create a new repo in GitHUB and then go settings -> webhooks and services

38.png

Click on add service and choose Jenkins GitHub Plugin

40.png

Now on your local machine create a new folder to create a new repo. You can follow the instructions from the newly created repo

43.png

Now clone the repo and empty folder in your computer

git clone https://guthub.com/*******************

44.png

 

PROJECT CREATION

Let’s go back and create a new project in Jenkins. On the main page click on new item -> FreeStyle Project. You will need to configure the following settings

  1. Click on restrict where this project can run. and from label use WindowsGroup. This makes sure this project runs only on windows node.
  1. The next step is to setup the Source Node Management and choose GIT. Type the Git repo URL 47
  2. Now you have to add your GIT credentials in Jenkins. Click on it and add your credentials 48.png41.png
  3. Go down to build and select Windows PowerShell51.png
  4. Add the following code

$count = (Invoke-ScriptAnalyzer -Path ‘C:\jenkins\workspace\PowerShell\*.ps1’)
$count1 = $count.count
if($count1 -gt 0) {
$LastExitCode = ‘2’
write-output $count
}

This code look for any .ps1 scripts on the folder. If any errors are found it will pass them Jenkins

Now lets test and see if can run a build. Click on Build

52.png

54.png

You can click on console output and see the log file and at the end will say success

55.png

You will see that Jenkins download data into the folder C:\jenkins\workspace\powershell

 

UPLOADING A SCRIPT TO GIT AND RUN A BUILD A JOB

Finally, we have gotten to the fun part. I am going to create a simple script .ps1 one liner that uses write-host. Script analyzer will flag the script

  1. Go to your GIT folder and create the .ps1 there 56.png
  2. Now we need to push the script. Type the following commands
    • git add –all
    • git commit -m “This is my first commit”
    • git push
  3. After that it should prompt you for your username and password58.png
  4. Check GitHub to make sure the script is there60
  5. Now click build now button on the project and lets see what happens61.png

You will see some failures and if you click on the project and then changes you will get commit message on the project

62.png

Now click on console output and you will see the errors, BINGO !!63.png

The message says avoid using Write-Host

6. Let’s fix the script now and instead of using write-host, I am going to use Write-output and then upload to GitHub

64

If you click on Build you will see that the project builds without problems

6566.pngYou can also tweak the job plan to check for changes on the Git repo automatically, or check every some often.

 

CONCLUSION

Now you may wonder what can you really do with this in your company or what benefits may have for you and your team

  • You can create, modify script Analyzer rules and by using a CI tool it will make sure everybody is using the same rules when uploading code to your repo.
  • When doing code review is one less human step in the way
  • You can have your CI as the deployment method to your systems. This means that no engineers  or developers can push scripts without some kind of control or central trigger
  • Time saved

What is next ?  Well after you have some kind of pipeline you can start adding pester tests to your job, automatic code rejection, test code on different windows operating systems and different WMF versions, etc.

If you are new to DevOps  movement you may wonder how is this DevOps; I can give you my personal definition based on these 3 pillars of DevOps practices

Automation -> Using PowerShell Scripts to make changes to the system

Sharing -> Using Git as your central repo; everybody has access to it and people can make changes to the existing code

Metrics-> You should get metrics on how long does it take to test, fix and deploy code manually. Then measure how long it takes the same process with a pipeline in place

 

 

 

 

 

 

 

 

 

 

7 thoughts on “CI using AWS, Jenkins, Git and PowerShell

  1. Good write-up.

    Jenkins is also available for Windows natively (and very stable), so if all your stuff is Windows-based and/or you’re cost-conscious, it might make sense to just have a Windows CI server, instead of a Jenkins server on Linux and a build slave on Windows.

    Like

  2. devopsconsult

    You have written a very nice article. You have a good command on Jenkins. Keep it up! I will follow up your blog for the future post.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s