A Tale of
Contemporary Software
UNSW Guest Lecture
March 2015
About Me
- Comp Science Bachelor UNSW
- Applied Finance PostGrad at Macquarie Uni
- IBM, Sun, Accenture, BGI/BlackRock, Macquarie Group
- Now innovating and opinionating at Trunk Platform
yunspace
yunzhilin
yunspace.com
- My opinions does not reflect the view of the company
- Shout out if you have questions or need a break
Challenges
Case Studies
Solution
Implement*
Challenges
Market Efficiency
-
We live in a Semi-Strong Efficiency market
-
Stock prices reflect and adjust to all public information
-
Little room for arbitrage or outrageous profit
-
-
Information Efficiency also applies to business & competition:
-
What you try to do, others are already doing/copying
-
Your customers are very likely to know more than you
-
You need to process information fast, but also relevant
-
Technology
becomes Key Differentiator
Superior technology gives competitive advantage:
-
Exploiting Market Anomalies such as Momentum trading
-
Arbitraging tiny inefficiencies: Quantitive, Frequency trading.
-
Destroying information asymmetry and "experts": RPdata
-
Disrupting established markets: Uber, Tesla
-
Providing better UX for boring things: CBA, Simple.com
-
All we need is good software ...
Every Company MUST be
Software Company
The era of separating traditional industries and technology industries is over—and those who fail to adapt right now will soon find themselves obsolete.
Now Every Company is A Software Company - Forbes 2011
- Offshoring and underinvestment in core technology is detrimental
- solution is not cost cutting, but more effective technology
- "tech refresh" is primitive idea, should always refresh
- CIO are supposed to be visionary leaders, not accountants
Conway's Law
organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations
Melvin Conway - 1968
Don't Forget Conway's Law - Sarah Novotny (NGinx)
Good Software Design starts off with Orgnisational Design
Siloed System
Information and Capabilities lost in translation between silos
Prefered System
Lead to architecture based around business capabilities
Software Systems must Connect
Information are rarely stored in a single place, but integration of systems is hard:
- Any two applications are different
- Application changes are inevitable
- Networks are unreliable and slow
Enterprise Integration Patterns, Gregor Hohpe
Conclusion
-
Technology can differentiate in a competitive world.
-
Challenge is to process information more efficiently
-
But building good software is hard:
-
Traditional siloed teams hinder collaboration
-
Integrating applications is painful
-
Case Studies
Aladdin Platform
- 25Million lines of code
- One stop shop for risk & portfolios
- Central DB, no integration needed
- 20,000 users, 30,000 portfolios
- 1000+ developers, built over 20 yrs
- single point of failure
- death by stored procedures
- slow change cycle (legacy)
- siloed development team
- thick client
Built around 1994. Iconic Monolithic, OS for traders
Tightly Coupled Systems
= Useless but Deadly Change Review Meetings
- Legacy monoliths tend to be defensive, "keep the lights on"
- Lack of tests -> unknown risks -> afraid to make changes
- Making changes is equivalent to "chucking a Prince Oberyn"
BGI Global Messsaging
- Publish / Subscribe
- Real time, high speed
- Across 3 time zones
- Canonical Data + Enrichment
- Decoupled end services
- custom 3rd party messaging
- ESB potential point of failure
- Data is Publisher driven
- Cross functional stops at ESB
- Bottleneck shift to ESB
- $$$ internal & external work
- Tragedy of the commons
Enterprise Service Bus
ETF
Cash
Fixed
Equities
Custodians
SWIFT
Counter
Party
early 2000s, Traditional SOA over ESB
ESB promises:
That's a lot! Concerned about Separation of Concerns?
Scaling the ESB
- One size fits all: Every on the Bus gets same treatment.
- Looser coupling and independent scaling would be nice
ESB in Reality
Conclusion
- Monoliths and tight coupling are bad
- ESBs isn't perfect, but stepping in right direction
Solution
40 years of Service Evolution
Microservices: The resurgence of SOA principles and an alternative to the monolith
- PWC Technology Forecast 2014
Change Management
Microservices: The resurgence of SOA principles and an alternative to the monolith
- PWC Technology Forecast 2014
What are MicroServices
- http://martinfowler.com/articles/microservices.html
- NetFlix, Amazon, realestate.com.au, Tyro, Atlassian
- Similar to traditional SOA, except no more ESB
- Consumer driven contracts instead of Canonical Data
- Fault tolerant and independently scalable
Activity Based Working
Self
Orgnising
Team
Cross Functional Delivery
Business
UI/UX Design
Front End Dev
Back End
Dev
Infrastructure/DevOps
Conclusion
- Split into Self Organised Teams around Capabilities
- Cross functional collaboration deliver Capabilities
- Decoupled and scalable Capabilities collectively form Microservices Architecture (MSA)
Implement
(optional homework)
Test Driven Development
-
How do you know when you broke something?
-
you don't, so write tests!
-
There is no such thing as untestable code @CodaHale
Write Clean Code
Highly recommend Uncle Bob's book:
- Clean Code
- The Clean Coder
Find out more at:
Twelve Factor App
By Heroku: 12factor.net
Some opinionated points that apply to University work:
- One Code base - use git: github/bitbucket
- Config - use environment variables
- Dependencies - use gradle to avoid death by xml:
repositories {
jcenter()
maven { url "http://dl.bintray.com/trunkplatform/osworkflow" }
}
}
dependencies {
compile group: 'com.trunkplatform', name: 'service', version: '3.1.4'
}
<repositories>
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>bintray-trunkplatform</id>
<name>bintray</name>
<url>http://dl.bintray.com/trunkplatform/osworkflow</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.trunkplatform.opensymphony</groupId>
<artifactId>service</artifactId>
<version>3.1.4</version>
<scope>compile</scope>
</dependency>
</dependencies>
Use REST, no WSDL/SOAP
{
"streetNumber": "80",
"streetName": "Clarence",
"suburb": "Sydney"
}
<?xml version="1.0"?>
<soap:Envelope
xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
<soap:Body xmlns:p="http://www.platmasphere.com/createProperty.xsd">
<p:CreateProperty>
<m:Property>
<m:streetNumber>80</m:streetNumber>
<m:streetName>Clarence</m:streetName>
<m:suburb>Sydney</m:suburb>
</p:Property>
</m:CrateProperty>
</soap:Body>
</soap:Envelope>
PUT /hostname/properties/
<definitions name="PropertyService"
targetNamespace="http://namespaces.snowboard-info.com"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns="http://schemas.xmlsoap.org/wsdl/">
<!-- not including model schemas -->
<message name="CreatePropertyRequest">
<part name="body" element="CreatePropertyRquestModel"/>
</message>
<message name="GetPropertyResponse">
<part name="body" element="CreatePropertyResponseModel"/>
</message>
<portType name="Properties_Port">
<operation name="CraeteProperty">
<input message="CreatePropertyRequest"/>
<output message="CreatePropertyResponse"/>
<fault message="CreatePropertyFault"/>
</operation>
</portType>
<binding name="CreateProeprtySoapBinding"
type="GreatePropertytType">
<soap:binding style="document"
transport="http://schemas.xmlsoap.org/soap/http"/>
<operation name="CreateProperty">
<soap:operation
soapAction="http://www.platmasphere.com/CreateProperty"/>
<input>
<soap:body use="literal"
namespace="http://platmasphere.com/CreateProperty.xsd"/>
</input>
<output>
<soap:body use="literal"
namespace="http://platmasphere.com/CreateProperty.xsd"/>
</output>
<fault>
<soap:body use="literal"
namespace="http://platmasphere.com/CreateProperty.xsd"/>
</fault>
</operation>
</binding>
<service name="Properties">
<documentation>WSDL File for PropertyService</documentation>
<port binding="CreateProeprtySoapBinding" name="Properties_Port">
<soap:address location="http://hostname/propeties/" binding="CreatePropertySoapBinding" />
</port>
</service>
</definitions>
- Managed to avoid death by XML again!
Good REST
is not
Remote Proc Calls over HTTP:
-
POST /api/getUserById/{id}
-
POST /api/createUser/
-
POST /api/updateUser/{id}
-
POST /api/delUserById/{id}
is
Resource oriented, http verbs:
-
GET /api/user/{id}
-
POST /api/user
-
DELETE /api/user/{id}
-
PUT /api/user/{id}
REST Verbs Reference:
REST in Java
- Java API for Restful Services: https://jax-rs-spec.java.net/
- Implemented by Jersey: https://jersey.java.net/
- Made even more awesome by: http://dropwizard.io/
+
=
PROFIT
Dropwizard.io
Dropwizard makes it easy to do the right thing, allowing you to concentrate on the essential complexity of a problem rather than the plumbing
Getting Started:
http://www.dropwizard.io/getting-started.html
Sample Project with Heroku support:
https://github.com/Trunkplatform/dropwizard-petstore
Our Dropwizard-Turbo LazyBones Template:
https://github.com/Trunkplatform/lazy-bones-dropwizard-turbo
PostMan
For manual testing: https://www.getpostman.com/
Swagger.io
- Who needs WSDL when you have Interactive Docs
Questions?
I'll be talking some more at:
- Dropwizard and Friends - Microservices Meetup 1st April
- www.trunkplatform.com
- Startup focusing on Real Estate
- Old industry, new ideas
- We are very Contemporary
- Self Organised
- And we use Microservices
We are Hiring!
A Tale of Contemporary Software
By Yun Zhi Lin
A Tale of Contemporary Software
UNSW Guest Lecture March 2015
- 7,149