Engineering@Nuxeo
Sneak peak in the sausage factory
![](https://s3.amazonaws.com/media-p.slid.es/uploads/101047/images/1725072/nx-font.png)
Thierry Delprat
tdelprat@nuxeo.com
https://github.com/tiry/
Nuxeo Engineering Team
People behind the code
![](https://s3.amazonaws.com/media-p.slid.es/uploads/101047/images/1725189/nx-background.002.png)
1 team - 360 view
the Nuxeo Engineering Team handles
-
development of new features in the product
-
design, development, testing
-
design, development, testing
-
customer support
-
reproduce, investigate, fix
-
reproduce, investigate, fix
-
consulting
- design workshops, training, project bootstrap ...
get feedback from customers
efficient support for
customer developers
always up-to-date with the platform
Distributed team
-
18 engineers in France
- 14 engineers in Paris
- 3 engineers in Lille
-
1 engineer in Lyon
-
7 engineers in the US
- 4 engineers in NYC
- 3 engineers in California
-
8 engineers in EU
- 7 engineers in Portugal
-
1 engineer in Germany
-
Product Team in Paris
- 1 Product Manager
- 1 Developer advocate
- 2 Technical writers
- 1 Technical consultant
Specialized SQUADS
-
dedicated support team
- 2 engineers in Paris
- 1 engineer in California
- 1 engineer in Iowa
-
consulting team
- 1 project manager in Paris
- 1 project manager in NYC
- 1 dedicated engineer in Paris
- all US engineers are mixed between Product/Project
Nuxeo Developer profile
-
Experienced and passionate developers
-
they want to do quality code
-
they will complain if something reduces quality
-
-
Expectations are high
-
Best practices to ensure QA and maintainability
-
Coding rules, Issues Management, CI, Code review
-
-
Doing bad code has direct consequences
-
Other developers get angry
-
Support duty becomes a burden
-
-
they defend the QA proces
Developers work hard to learn and adapt or they leave quickly
Nuxeo DeveloperS
- Integrate new developers
-
6 new developers in the last 12 months
-
6 new developers in the last 12 months
-
Keep the good developers
-
in average developers are with us for 5+ years
-
-
Keep growing expertise
-
our 8 top experts are with us for 10+ years
-
Work Organization
- most developers are grouped in 4 teams
- work on related tasks in iterations
-
Scrum boards (in JIRA)
-
some experts developers work alone
-
focus on complex technical tasks
-
Kanban style or Scrum board (Jira)
-
- Sprint Planning / Sprint review every 2 or 3 weeks
- Backlogs filled conjointly
- Product Manager & Head of Engineering
- Head of Support
- CTO
Technical coordination: Techleads
- Technical leadership
- each component of the platform has an owner
-
each team as a Technical Lead
- Technical leads
- are responsible for proposing design
-
are responsible for raising issues
- Techleads meeting every 2 weeks
- discuss issues and define solutions
- prioritize infrastructure tasks (technical debt)
- feed backlogs
No StupiP coding
- Developers are responsible
- for good design
- for QA
- for performances
-
for handing support
- Techleads are here to help
- review design
- review code
- discuss performances
Technical CommunicaTION
- Standup meeting 3 times a week
-
share day to day tasks and issues
-
share day to day tasks and issues
- Slack Channels
- discuss about issues
- share new ideas or technologies
-
intra-team channels
- Techleads reports
- share ongoing tasks, issues & concerns
-
share knowledge
Coding @ Nuxeo
Principles and Rules
![](https://s3.amazonaws.com/media-p.slid.es/uploads/101047/images/1725189/nx-background.002.png)
QA & Testing part of our DNA
-
TDD by nature
- we were already doing TDD in 2005
-
we have build a versatile Unit Testing Framework
- no excuse
-
Testing at all levels
- Unit tests
- Functional testing
- Performances testing
-
CI supports the testing work
- each developer has his own Test&Push job
- Jenkins runs the performances and functional tests
- Jenkins runs the matrix testing (OS / DB)
History and Code Tracking
-
Task tracking
- All issues are created in JIRA
- Issues are assigned to Iteration & Agile Boards
- Status indicate progress
-
Tasks links associated commits
-
Commit History
-
Every single commit is associated with a Jira Issue
- tracability (support)
-
Git History is clean
- re-base policy
- feature development is done in separated branches
-
Every single commit is associated with a Jira Issue
Coding Rules & Best Practices
-
Nuxeo Core Developer Guide is public
-
This guide includes
-
guidelines for using all the tools
-
Maven, Git, Npm, Bower ...
-
-
coding rules
-
guidelines for avoiding security issues
-
-
Code analysis
-
For now the most efficient remains Code review by peers
-
we have a lot of experienced developers
-
a lot of them review commits and raise hand when something is wrong
-
-
CI & DEvops
-
Jenkins is used extensively
- JUnit, WebDriver & Selenium testing
- Gatling & FunkLoad testing
- Packaging (jar, Nuxeo Packages, Distributions ...)
-
Deployment (Maven, Marketplace, Connect, Production)
-
DevOps teams helps maintaining the process
- Sys-admin + developer skills (3 persons)
CI Role
![](https://www.lucidchart.com/publicSegments/view/fddb78d5-97c5-4f10-83b6-94085ff8430c/image2.png)
some Figures
- 2+ Million lines of code
- 15,000 commits
- 500+ Jenkins Jobs declared
- 3,000+ build per 24h
- 500,000 tests run per 24h
- Standard Build = 5,500 unit tests + 6,000 functional tests
- 15,000 NXP Tasks completed
- 15,000 SUPNXP Issues solved
![](https://s3.amazonaws.com/media-p.slid.es/uploads/101047/images/1725193/nx-logo.png)
Source Code Management
Releases cycle, Feature branches, HotFixes ...
![](https://s3.amazonaws.com/media-p.slid.es/uploads/101047/images/1725189/nx-background.002.png)
![](https://www.lucidchart.com/publicSegments/view/6d7f106f-99b8-47f0-9557-72968280381f/image.png)
![](https://www.lucidchart.com/publicSegments/view/b1c66072-7841-4d7a-aeda-bee209634ed4/image.png)
Release a new FastTrack every 2 to 3 months
![](https://www.lucidchart.com/publicSegments/view/707d1cbf-df57-43f2-ba85-4d3e457ca40b/image.png)
Release a new FastTrack every 2 to 3 months
Give early access to new features for ongoing projects
![](https://www.lucidchart.com/publicSegments/view/3cb3dc35-ebc5-4a58-a47e-fc962d424731/image.png)
4 to 5 FastTrack releases per year
![](https://www.lucidchart.com/publicSegments/view/ef399450-fbf7-45de-99fa-4fdea3db2d4d/image.png)
Release LTS once a year
![](https://www.lucidchart.com/publicSegments/view/97b253bf-e86e-4263-8a99-2caebbd4e7ee/image.png)
LTS Long Term Support
![](https://www.lucidchart.com/publicSegments/view/97b253bf-e86e-4263-8a99-2caebbd4e7ee/image.png)
LTS Maintenance Branch
![](https://www.lucidchart.com/publicSegments/view/c9d6d26d-1444-454d-9952-d89db5b53a8d/image.png)
New Features are developed
in isolated "feature branch"
we can work on several features at the same time without any interactions
![](https://www.lucidchart.com/publicSegments/view/c9d6d26d-1444-454d-9952-d89db5b53a8d/image.png)
CI does continuous testing of the branch
detect issues early
![](https://www.lucidchart.com/publicSegments/view/5b17eb1a-63d7-4ec5-897f-a850dc9393dd/image.png)
when ready changes are integrated in a
Pull Request
![](https://s3.amazonaws.com/media-p.slid.es/uploads/101047/images/2575270/PR-simple.png)
![](https://www.lucidchart.com/publicSegments/view/df9ab984-97cb-48d1-9045-8f2bdb904886/image.png)
Pull Request is reviewed by peers
![](https://s3.amazonaws.com/media-p.slid.es/uploads/101047/images/2575274/PRNinja.png)
![](https://www.lucidchart.com/publicSegments/view/fcdc2e4a-f244-4f70-a293-ce9c10d6cb7a/image.png)
when changes are accepted
the new feature is merged
feature branch
is automatically deleted
![](https://s3.amazonaws.com/media-p.slid.es/uploads/thierrydelprat/images/707027/jenkins.jpg)
![](https://www.lucidchart.com/publicSegments/view/1b9bf808-ebfe-4d50-9694-95618a404fa2/image.png)
Fixes are done in a branch too
![](https://www.lucidchart.com/publicSegments/view/409d408d-007a-4778-bc84-f7cb81069349/image.png)
Fixes benefit from CI Services too
including PR & Review
![](https://www.lucidchart.com/publicSegments/view/6f7622e5-13fb-4c9d-9811-c608452986d5/image.png)
Fixes may be back-ported to LTS
(Cherry pick)
![](https://www.lucidchart.com/publicSegments/view/a3cbb3f7-dea3-430a-a635-cc2bb69e1512/image.png)
Backported-Fixes are tested
![](https://www.lucidchart.com/publicSegments/view/33741d5d-85e4-430f-820d-df3be0f836b3/image.png)
Backported-Fixes
are reviewed
![](https://www.lucidchart.com/publicSegments/view/33d42880-7afa-463d-abb0-5c23fce0341a/image.png)
Backported-Fixes
are reviewed
![](https://www.lucidchart.com/publicSegments/view/759d43a4-fe72-444f-9056-901ccb2ae296/image.png)
Backported-Fixes
eventually become
Hot Fixes packages
About HotFixes
Released every 2 to 4 weeks for each of the 3 LTS
- quickly integrate fixes required by customers
- unlock new configurations option when needed
- make LTS benefit from performances improvements
34 HotFixes released in 2015
- LTS 2015 - 2 (LTS is released in October)
- LTS 2014 - 21
- LTS 2013 - 9
- LTS 2012 - 2
Branches & dev cycles
By Thierry Delprat
Branches & dev cycles
- 2,241