Continuous Integration and Delivery

INSA - LYON - 4th of JUNE, 2014


ABOUT HENRI GOMEZ


TODAY - CI Architect @ Axway*   

Ops Hat - Former Senior Ops Director
Dev Hat - FORMER Dev, Team Leader and Architect
QA Hat - FORMER QA Tech Leader

JPackage* OBuildFactory* anD DEVOPS incubator* PROJECT FOUNDER
PROUD MEMBER OF ASF (Tomcat* & XML-RPC*) &  OSSGTP*
LONG TIME OSS ACTIVIST 

ABOUT Axway



Axway, a market leader in governing the flow of data, is a global software and services company with more than 11,000 public- and private-sector customers in 100 countries. For more than a decade, Axway has empowered the world’s top organizations with proven solutions that help manage business-critical interactions through the exchange of data flowing across the enterprise, B2B communities, the cloud and mobile devices.
Axway is positioned as a Leader in both the Gartner 2013 Magic Quadrant for On-Premises Application Integration Suites (Axway 5 Suite), and the Magic Quadrant for Application Services Governance (Axway API Server).

TODAY Software Challenges


Software projects are larger and complex
More and more component interoperability
Many teams and often remote teams
Tight schedules
Frequent needs and requirements changes

AGILE METHODOLOGY

AGILE IDEAL CYCLE


CONTINUOUS INTEGRATION


"Continuous Integration is the practice of integrating early and often, so as to avoid the pitfalls of integration hell".

CI IS TOOLING OF AGILITY


Standardize Build, Test and Deployment
Provide Continuous Builds, Test and Deployment
Ensures solid, reproducible and analyzable steps
Visible to all actors (Dev, QA, Product Owner)

GOAL


THE ULTIMATE GOAL IS TO REDUCE TIMELY REWORK AND THUS REDUCE COST AND TIME.
 
WHEN DONE WELL, CONTINUOUS INTEGRATION HAS BEEN SHOWN TO ACHIEVE THESE GOALS.

CONTINUOUS INTEGRATION STEPS


Fetch From SCM
Build Code
Run Unit Tests
Run Integration Tests
Deploy Artifacts
Run Acceptance Tests
Produce Reports

HUMAN WORKLOAD


Team members work on code

code
tests DESIGN
docs

Automated workload


POLL SCM AND CHECK OUT IF CHANGES
BUILD CODE 
RUN UNIT & INTEGRATION TESTS
DEPLOY BINARY CONTENTS TO ARTIFACTS REPOSITORY
RUN ACCEPTANCE TEST
PRODUCE REPORTS

regarding tests

TESTS SHOULD COVER A WIDE SCOPE FROM DEV UP TO RUNTIME

UNIT TESTING
INTEGRATION TESTING
ACCEPTANCE TESTING
PERFORMANCES TESTING

UNIT TESTING


HERE WE TEST CODE
Ensures the code is doing what we intend it to do
Many extremely quick and small tests

UNIT TESTING ensure we make the software right

INTEGRATION TESTING


USED TO VALIDATE CONTRACTS BETWEEN PARTICIPANTS
SHOULD BE PORTABLE (NOT PLATFORM DEPENDANTS)
"External Components" SHOULD BE embeddedABLE or mocked 

INTEGRATION TESTING ensure we make the right software

ACCEPTANCE TESTING


KEEP THEM SIMPLE
REAL TARGET COMPONENTS SHOULD BE USED
PREFER PRE-REQUISITE (DONT EMBED SQL DB ENGINES)
DATASETS IN USE SHOULD BE MODERATE

PREPARE FOR PRODUCTION - ENROLL YOUR OPS

PERFORMANCE TESTING


Fix quickly performance regressionS
Prepare for Capacity Planning 

proTECT YOURSELF FROM IN-PROD DISCOVERY

CONTINUOUS DEPLOYMENT

CONTINUOUS DEPLOYMENT IS CONTINUOUS INTEGRATION ON STEROIDS

CONTINUOUS DEPLOYMENT FOR ?


YOUR DEVs 
YOUR QAs
YOUR SUPPORT
YOUR SALES
AND OF COURSE YOUR CUSTOMERS

ANY MAGIC RECIPES ?


KEEP IT SIMPLE

THINK COMPONENT

ENROLL YOUR OPS !

A WORKING RECIPE - LEGO MODE


Assemble BINARY Artifacts TO CREATE RUNTIME COMPONENTS
EXAMPLE : A WEB APPLICATION
 
JAR + WAR + TOMCAT = runtime
VM + RUNTIME = SERVICE

ASSEMBLE THEM ALL


DEPLOY USABLE ARTIFACTS (RUNTIME)
READY TO USE BY UPSTREAM TEAMS LIKE QA-OPS (VM/RPM/DEB/MSI/NUGET)
IT SHOULD BE CONFIGURABLE FROM THE OUTSIDE (HARDCODING IS EVIL)

DELIVERY AND DEPLOYMENT


THERE IS NEEDS FOR CONTINUOUS DELIVERY
AND CONTINUOUS DEPLOYMENT

DELIVERY


ProvidE product to end user
User could then install or update it on his own systems
SOFTWARE VENDORS ARE IN "CONTINOUS' DELIVERY MODE

DEPLOYMENT


PROVIDE OPERATIONAL SOLUTION TO CUSTOMERS
CUSTOMERS COULD THEN DIRECTLY USE IT ON HIS OWN OR LEASED SYSTEMS
SERVICE PROVIDERS ARE IN CONTINUOUS DEPLOYMENT LAND (CLOUD)

CD : EXTERNAL APPROACH


Many good providers, Cloud Based



Cloud bees*, heroku*, openshift*

BUT you MAY don't have choiceS and HAD do it IN-HOUSE

CD : In-HOUSE APPROACH


Materialize 3 chains


SOFTWARE FACTORY
QUALITY FACTORY 
DELIVERY&DEPLOYMENT FACTORY

AND DONT FORGET SERIOUS MONITORING FOR ALL OF THEM

SOFTWARE FACTORY


FROM SCM TO ARTIFACTS


BUILDs
UNIT TESTS
INTEGRATION TESTS
CODE QUALITY REPORTS (SONAR)
DEPLOY BINARY ARTIFACTS

QUALITY FACTORY


FROM ARTIFACTS TO QA/RUNTIME REPORTS


CONSUME ARTIFACTS FROM SOFTWARE FACTORY
RUN ACCEPTANCE TESTS
RUN PERFORMANCE TESTS AND CAPACITY PLANNING
GENERATE REPORTS

DELIVERY & DEPLOYMENT FACTORY

FROM ARTIFACTS TO RUNTIME


BUILD NATIVE PACKAGES (RPM/DEB/MSI/NUGET)
BUILD IMAGES (VMDK, AMI, DVD ISO)
PROVIDE RUN INFRASTRUCTURE (INSITE)
INTERFACE TO RUN INFRASTRUCTURE (OFF-SITE/PUBLIC)
CATALOG & PROVISION RUNTIMES

INTERACTIONS BETWEEN FACTORIES


SOFTWARE FACTORY PRODUCES SOFTWARE MATERIALS 
QUALITY FACTORY CONSUMES SOFTWARE MATERIALS
QUALITY FACTORY GIVES 'GO FOR DELIVERY' TO D&DF
D&DF PROVIDES  QUALITY FACTORY RUN INFRASTRUCTURE

ENCOUNTERED TOOLING


SCM : SUBVERSIOn, GIT, MERCURIAL (... or CVS)
ORCHESTRATiON : JENKINS* (What else ?)
CONTENT REPOSITORIES : ARCHIVA*, ARTIFACTORY*, NEXUS*, HTTPD*
CODE QUALITY : SONAR*
WEB TESTING : SELENIUM*

ENCOUNTERED TOOLING 


ACCEPTANCE : THUCYDINES*
PERFORMANCE : GATLING* 
RUN FORCE : JENKINS AGENTS
HARD MIDDLEWARE : ESXi*,  OPENSTACK*/CLOUDSTACK*
SOFT MIDDLEWARE : DOCKER*, LXC*, VirtualBOX*, VAGRANT*
MONITORING : Zabbix*, GRAPHITE*, COLLECTD*, JMXTRANS*

TO CONCLUDE




MANDATORY

FUN

SOCIAL



CI & CD ARE MANDATORY TODAY


PROVIDES AGILE TOOLING
ACCELERATE SOFTWARE PRODUCTION
SECURE SOFTWARE DELIVERY


CI & CD ARE FUN


COVERS A WIDE SCOPE FROM DEV UP TO OPS
LARGE SCALE OF TECHNOLOGIES
VERY ACTIVE ECOSYSTEMS AND COMMUNITIES
Always something new to learn and PUT IN PLACE


CI & CD ARE SOCIAL


WORKING WITH DEV, QA, OPS AND SALES TEAMS,  CI & CD ACT AS FACILITATORS



THANK YOU


Made with Slides.com