Deployment

using Jenkins on AWS

Who am I?

Manuel Kanah

manuel@kanah.it

twitter: @testinaweb

Github: testinaweb

Deployment

A deployment process consists of several interrelated activities with possible transitions between them.

Common issues

• Testing
• Versioning/Tagging
• Consistency

Testing

Tests in general are the way to ensure the quality of the code.

Versioning

You want to versionate your software releases.

Every times you deploy your code you need to pin/tag the specific version of your code.

Consistency

Your code has to be delivered to all your machines

or nowhere.

AWS issues

If you want to deploy using Jenkins you must deal with:

AUTOSCALING

AWS solution

AWS CLI

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

Solution using Jenkins

Jenkins is able to do a lot of things.

It is very good on bases operation out of the box.

We need deep configuration, "Git plugin" and

a good shell script.

Jenkins needs

• EC2 needs role
  • S3
  • EC2
  • ELB
  • AUTOSCALING
• Configure the Security Group
• Jenkins needs the PublicKey.pem
• .ssh/config
  • StrictHostKeyChecking no
  • UserKnownHostsFile=/dev/null
• Git
• PHP (composer)

Deployment flow

• Blank the Jenkins' workspace
• Download repository
• Download dependencies
• Run tests
• Version the release (on S3)
• Stop autoscaling group
• Retrieve the details of all ec2 instances
• Iterate over each ec2 and deploy the release
• Start autoscaling group

Everything starts here

Handle the code

if [  "${NEW_RELEASE}" == 'true' -a ${ROLLBACK} -gt 0 ]
then
    exit 1
fi;

if [ "${NEW_RELEASE}" == 'true' ]
then
    # code preparation
    if [ -d "${WORKSPACE}/vendor" ]
    then
        echo 'Removing vendor directory'
        rm -rf ${WORKSPACE}/vendor
    fi;

    # tests and QA

    if [ -f "${WORKSPACE}/composer.json" ]
    then
        echo 'composer.json is found'
        echo 'Running composer install'
        /usr/local/bin/composer install --no-dev --optimize-autoloader
    fi;

    find . -name ".git*" -exec rm -rf {} \;

    local_tar_filename=${KEY_WORD}.${BUILD_NUMBER}.$(date +%Y%m%d).$(date +%H%M%S).tar.gz
    s3_tar_filename=s3://${AWS_S3_BUCKET}/${KEY_WORD}/$(date +"%Y")/$(date +"%m")/${local_tar_filename}
    tar -cvzf ${local_tar_filename} -C ${WORKSPACE} . --exclude ${local_tar_filename}

    aws s3 --profile s3 cp ${local_tar_filename} ${s3_tar_filename}
fi;

if [ ${ROLLBACK} -gt 0 ]
then
    tail_rollback=$((${ROLLBACK} + 1));
    release="s3://${AWS_S3_BUCKET}/$(aws s3 --profile s3 ls s3://${AWS_S3_BUCKET}/${KEY_WORD}/ --recursive | awk '{print $4}' | \
        sort -V | tail -n ${tail_rollback} | head -n 1)"
else
    release="s3://${AWS_S3_BUCKET}/$(aws s3 --profile s3 ls s3://${AWS_S3_BUCKET}/${KEY_WORD}/ --recursive | awk '{print $4}' | sort -V | tail -n 1)"
fi;

if [ "${release}" == "s3://${AWS_S3_BUCKET}/" ]
then
    exit 1
fi;

Handle Autoscaling

# stop autoscaling
scaling_group=$(aws autoscaling describe-auto-scaling-groups --profile ${profile} | grep -i -P "(?=.*?${AUTOSCALING_GROUP_NAME})" | \
    grep -i -v qa | head -n 1 | awk '{print $2}' | cut -d '"' -f 2)
aws autoscaling suspend-processes --profile ${profile} --auto-scaling-group-name ${scaling_group}

elb_name=$(aws elb describe-load-balancers --profile ${profile} | grep LoadBalancerName | grep -i ${LOAD_BALANCER_NAME} | \
    grep -i -v qa | sed -E 's/(.*):{1}\ "(.*)",/\2/')

ec2_instances=$(aws ec2 describe-instances --profile ${profile} \
    --filters "Name=tag:Name,Values=${EC2_TAG_KEY_NAME}" "Name=instance-state-code,Values=16" \
    --query "Reservations[*].Instances[*].[PrivateIpAddress,InstanceId]" \
    --output text)

printf "%s\n" "${ec2_instances}" | while IFS=$'\t' read -r ip_address instance
do
    if [ "${SET_OF_EC2_INSTANCES}" != '' ]
    then
        skip_instance="true"
        while read -r instance_id
        do
            if [ "${instance_id}" == "${instance}" ]
            then
                skip_instance="false"
                break
            fi;
        done < <(printf "%s\n" "${SET_OF_EC2_INSTANCES}")
        if [ "${skip_instance}" == "true" ]
        then
            continue
        fi;
    fi;

    aws elb deregister-instances-from-load-balancer --profile ${profile} --load-balancer-name ${elb_name} --instances ${instance}

    rsync -avL -e "ssh -i ${JENKINS_HOME}/PublicKey.pem -o 'StrictHostKeyChecking no'" "$(basename ${release})" ec2-user@${ip_address}:
    ssh -i ${JENKINS_HOME}/PublicKey.pem -o "StrictHostKeyChecking no" -tt ec2-user@${ip_address}  << 'EOF'
# deploy the code inside the EC2 instance
exit
EOF

    aws elb register-instances-with-load-balancer --profile ${profile} --load-balancer-name ${elb_name} --instances ${instance}
done;

# start autoscaling
aws autoscaling resume-processes --profile ${profile} --auto-scaling-group-name ${scaling_group}

New EC2

At the end of the provisioning of the EC2 instances, deploy the code with an API call:

# Push of the application
curl -X POST http://jenkins.domain.internal:8080/job/project-deployment/build \
    --data-urlencode \
    json="{\"parameter\": [{\"name\":\"BRANCH\", \"value\":\"origin/master\"}," \
    "{\"name\":\"SET_OF_EC2_INSTANCES\", \"value\":\"${INSTANCE_ID}\"}]}"

Other Solutions

• Code deploy
• Code Pipeline
• Cloud Formation
• Elastic Beanstalk

Manuel Kanah

http://goo.gl/QpdOLg