A Tale of

Contemporary Software

UNSW Guest Lecture

March 2015

About Me

  • Comp Science Bachelor UNSW
  • Applied Finance PostGrad at Macquarie Uni
  • IBM, Sun, Accenture, BGI/BlackRock, Macquarie Group
  • Now innovating and opinionating at Trunk Platform

yunspace

yunzhilin

yunspace.com

  • My opinions does not reflect the view of the company
  • Shout out if you have questions or need a break

Challenges

Case Studies

Solution

Implement*

Challenges

Market Efficiency

  • We live in a Semi-Strong Efficiency market

    • Stock prices reflect and adjust to all public information

    • Little room for arbitrage or outrageous profit

  • Information Efficiency also applies to business & competition:

    • What you try to do, others are already doing/copying

    • Your customers are very likely to know more than you​​

    • You need to process information fast, but also relevant

Technology

becomes Key Differentiator 

Superior technology gives competitive advantage:

  • Exploiting Market Anomalies such as Momentum trading

  • Arbitraging tiny inefficiencies: Quantitive, Frequency trading.

  • Destroying information asymmetry and "experts": RPdata 

  • Disrupting established markets: Uber, Tesla

  • Providing better UX for boring things: CBA, Simple.com

  • All we need is good software ...

Every Company MUST be

Software Company

The era of separating traditional industries and technology industries is over—and those who fail to adapt right now will soon find themselves obsolete.

Now Every Company is A Software Company - Forbes 2011

  • Offshoring and underinvestment in core technology is detrimental
  • solution is not cost cutting, but more effective technology
  • "tech refresh" is primitive idea, should always refresh
  • CIO are supposed to be visionary leaders, not accountants

Conway's Law

organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations

                                                              Melvin Conway - 1968

Don't Forget Conway's Law - Sarah Novotny (NGinx)

MelvinConway.com

Good Software Design starts off with Orgnisational Design

Siloed System

Information and Capabilities lost in translation between silos

Prefered System

Lead to architecture based around business capabilities

Software Systems must Connect

Information are rarely stored in a single place, but integration of systems is hard:

  • Any two applications are different
  • Application changes are inevitable
  • Networks are unreliable and slow

Enterprise Integration Patterns, Gregor Hohpe

Conclusion

  • Technology can differentiate in a competitive world.

  • Challenge is to process information more efficiently 

  • But building good software is hard:

    • Traditional siloed teams hinder collaboration

    • Integrating applications is painful

Case Studies

Aladdin Platform

  • 25Million lines of code
  • One stop shop for risk & portfolios 
  • Central DB, no integration needed
  • 20,000 users, 30,000 portfolios
  • 1000+ developers, built over 20 yrs
  • single point of failure
  • death by stored procedures
  • slow change cycle (legacy)
  • siloed development team
  • thick client

Built around 1994. Iconic Monolithic, OS for traders

Tightly Coupled Systems

= Useless but Deadly Change Review Meetings

  • Legacy monoliths tend to be defensive, "keep the lights on"
  • Lack of tests -> unknown risks -> afraid to make changes
  • Making changes is equivalent to "chucking a Prince Oberyn"

BGI Global Messsaging

  • Publish / Subscribe
  • Real time, high speed
  • Across 3 time zones
  • Canonical Data + Enrichment
  • Decoupled end services
  • custom 3rd party messaging

 

  • ESB potential point of failure
  • Data is Publisher driven
  • Cross functional stops at ESB
  • Bottleneck shift to ESB
  • $$$ internal & external work
  • Tragedy of the commons 

Enterprise Service Bus

ETF

Cash

Fixed

Equities

Custodians

SWIFT

Counter

Party

early 2000s, Traditional SOA over ESB

ESB promises:

That's a lot! Concerned about Separation of Concerns?

Scaling the ESB

  • One size fits all: Every on the Bus gets same treatment.
  • Looser coupling and independent scaling would be nice

ESB in Reality

Conclusion

  • Monoliths and tight coupling are bad
  • ESBs isn't perfect, but stepping in right direction

Solution

40 years of Service Evolution

Microservices: The resurgence of SOA principles and an alternative to the monolith

- PWC Technology Forecast 2014

Change Management

Microservices: The resurgence of SOA principles and an alternative to the monolith

- PWC Technology Forecast 2014

What are MicroServices

  • Similar to traditional SOA, except no more ESB
  • Consumer driven contracts instead of Canonical Data
  • Fault tolerant and independently scalable

Activity Based Working

Self

Orgnising

Team

Cross Functional Delivery

Business

UI/UX Design

Front End Dev

Back End

Dev

Infrastructure/DevOps

Conclusion

  • Split into Self Organised Teams around Capabilities
  • Cross functional collaboration deliver Capabilities
  • Decoupled and scalable Capabilities collectively form Microservices Architecture (MSA)

Implement

(optional homework)

Test Driven Development

  • How do you know when you broke something?

  • you don't, so write tests!

  • There is no such thing as untestable code @CodaHale

Write Clean Code

Highly recommend Uncle Bob's book:

  • Clean Code
  • The Clean Coder

 

Find out more at:

Twelve Factor App

By Heroku: 12factor.net

Some opinionated points that apply to University work:

  • One Code base - use git: github/bitbucket
  • Config - use environment variables
  • Dependencies - use gradle to avoid death by xml:
repositories {
    jcenter()
    maven { url "http://dl.bintray.com/trunkplatform/osworkflow" }
  }
}

dependencies {
  compile group: 'com.trunkplatform', name: 'service', version: '3.1.4'
}
<repositories>
    <repository>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
        <id>bintray-trunkplatform</id>
        <name>bintray</name>
        <url>http://dl.bintray.com/trunkplatform/osworkflow</url>
    </repository>
</repositories>

<dependencies>
    <dependency>
        <groupId>com.trunkplatform.opensymphony</groupId>
        <artifactId>service</artifactId>
        <version>3.1.4</version>
        <scope>compile</scope>
    </dependency>
</dependencies>

Use REST, no WSDL/SOAP

{
  "streetNumber": "80",
  "streetName": "Clarence",
  "suburb": "Sydney"
}
<?xml version="1.0"?>
<soap:Envelope
xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">

<soap:Body xmlns:p="http://www.platmasphere.com/createProperty.xsd">
  <p:CreateProperty>
      <m:Property>
      <m:streetNumber>80</m:streetNumber>
      <m:streetName>Clarence</m:streetName>
      <m:suburb>Sydney</m:suburb>
    </p:Property>  
  </m:CrateProperty>
</soap:Body>
</soap:Envelope>
PUT /hostname/properties/
<definitions name="PropertyService"
  targetNamespace="http://namespaces.snowboard-info.com"
  xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  xmlns="http://schemas.xmlsoap.org/wsdl/">

  <!-- not including model schemas -->

  <message name="CreatePropertyRequest">
    <part name="body" element="CreatePropertyRquestModel"/>
  </message>
  <message name="GetPropertyResponse">
    <part name="body" element="CreatePropertyResponseModel"/>
  </message>

  <portType name="Properties_Port">
    <operation name="CraeteProperty">
      <input message="CreatePropertyRequest"/>
      <output message="CreatePropertyResponse"/>
      <fault message="CreatePropertyFault"/>
    </operation>
  </portType>

  <binding name="CreateProeprtySoapBinding"
           type="GreatePropertytType">
    <soap:binding style="document"
                  transport="http://schemas.xmlsoap.org/soap/http"/>
    <operation name="CreateProperty">
      <soap:operation
        soapAction="http://www.platmasphere.com/CreateProperty"/>
      <input>
        <soap:body use="literal"
          namespace="http://platmasphere.com/CreateProperty.xsd"/>
      </input>
      <output>
        <soap:body use="literal"
          namespace="http://platmasphere.com/CreateProperty.xsd"/>
      </output>
      <fault>
        <soap:body use="literal"
          namespace="http://platmasphere.com/CreateProperty.xsd"/>
      </fault>
    </operation>
  </binding>

  <service name="Properties">
    <documentation>WSDL File for PropertyService</documentation>
    <port binding="CreateProeprtySoapBinding" name="Properties_Port">
        <soap:address location="http://hostname/propeties/" binding="CreatePropertySoapBinding" />
    </port>
  </service>

</definitions>
  • Managed to avoid death by XML again!

Good REST

is not

Remote Proc Calls over HTTP:

  • POST /api/getUserById/{id}

  • POST /api/createUser/

  • POST /api/updateUser/{id}

  • POST /api/delUserById/{id}

is 

Resource oriented, http verbs:

  • GET /api/user/{id}

  • POST /api/user

  • DELETE /api/user/{id}

  • PUT /api/user/{id}

REST Verbs Reference:

http://www.restapitutorial.com/lessons/httpmethods.html

REST in Java

  1. Java API for Restful Services: https://jax-rs-spec.java.net/
  2. Implemented by Jersey: https://jersey.java.net/
  3. Made even more awesome by: http://dropwizard.io/ 

+

=

PROFIT

Dropwizard.io

Dropwizard makes it easy to do the right thing, allowing you to concentrate on the essential complexity of a problem rather than the plumbing

ThoughtWorks Technology Radar

Getting Started: 

http://www.dropwizard.io/getting-started.html

Sample Project with Heroku support: 

https://github.com/Trunkplatform/dropwizard-petstore

Our Dropwizard-Turbo LazyBones Template:

https://github.com/Trunkplatform/lazy-bones-dropwizard-turbo

PostMan

For manual testing: https://www.getpostman.com/

Swagger.io

  • Who needs WSDL when you have Interactive Docs

Questions?

I'll be talking some more at:

  • www.trunkplatform.com
  • Startup focusing on Real Estate
  • Old industry, new ideas
  • We are very Contemporary
  • Self Organised
  • And we use Microservices

We are Hiring!

Made with Slides.com