Spec

End to end prototype of a Bill Aggregator that mimics vendor API. See below for detail specifications.

Data Model

• Providers - bill provider

FK: List(Subscriptions)

FK: List(Subscriptions)

FK: Provider.id, Premise.id, Bill.accountNumber, List(Bill.id, Status)

FK: Provider.id, Subscription.referenceNumber

FK: Provider, Premise, Subscription

Actors

• Producer - API hosted on Google Compute Engine
• API Operations - Insert, List, Get, Remove, Path on all entities, except Notification and Bill.

• Consumer - Use Postman HTTP client
• Sample Data - JSON files provided

Requirements

1. HTTP transport
2. JSON payload
3. Use compute engine
4. Authentication
5. API versioning
6. Ops-friendly
7. Data Validation
8. Batch considerations

Design

Frameworks Choice

• Node.js - full stack JSON, but lacking expertise
• Play! Framework - too complex for this exercise
• Dropwizard - simple, war-less and proven

Database Choice

• Google Cloud Datastore - lacking expertise
• MongoDB - perfect for JSON

Dropwizard + MongoDB = Meet Requirements

1. HTTP - Jersey RESTful web services
2. JSON - Jackson with Mongo-Jack, fastest Java parser
3. Use compute engine - Both API jar and DB hosted on GCE
4. Authentication - OAuth2 or custom Providers
5. API versioning - annotation based url mappings
6. Ops-friendly - healchecks and Codahale metrics
7. Data Validation - Hibernate Validatior
8. Batch considerations - use JSON lists

Plan

Due to the time contraint, it's important to decide on the approach taken to plan out a successfull prototype:

• Start to End - build each component fully one by one, can only partially demonstrate the story
• Minimum Viable Product - selectively build only critical operations for all components. So that together they tell a full story

Scope Adjustment

• Authentication - excluded
• API versioning - excluded
• Batch - simplified to single JSON parsing
• Queuing - another consideration for the real world, excluded
• Notification - manually triggered.

Final Use Case

1. create a subscriptoin

PUT /subscription/ 

2. Manually trigger notification, where UrlForBillRetrieval = subscription.callbackURL + billID

GET /subscription/trigger/{subscriptoinID}/{billID}/

3. retrieve a Bill

GET /bill/?billID={billID}

Implementation

1. Build Tool
2. Configuration
3. MongoDB
4. Healthcheck
5. JSON POJOs
6. Resources
7. Application Glue

Build Tool

Gradle combines the best of Ant and Maven

repositories {
    mavenLocal()
    mavenCentral()
}
project.ext {
    dropwizardVersion = '0.7.0-SNAPSHOT'
    mongojackVersion = '2.0.0-RC5'
    dropWizardConfig = './src/main/resources/billproto.yml'
}
dependencies {
    compile 'io.dropwizard:dropwizard-core:' + dropwizardVersion
    compile 'org.mongojack:mongojack:' + mongojackVersion
    testCompile group: 'junit', name: 'junit', version: '4.11'
}							
run {
    args 'server', dropWizardConfig
}

Configuration

• YAML

mongo:
  host:     ds039487.mongolab.com
  port:     39487
  db:       tycoon-mongo
  user:     ####
  password: ####
  							

public class MongoConfiguration {
    @NotNull
    public String host;
    @Min(1)
    @Max(65535)
    @NotNull
    public int port;
    @NotNull
    public String db;
    @NotNull
    public String user;
    @NotNull
    public String password;
}
							

MongoDB

• Hosting - MongLab GCE option
• Connection

mongo = new Mongo(mongoConfig.host, mongoConfig.port);
db = mongo.getDB(mongoConfig.db);
db.authenticate(mongoConfig.user, mongoConfig.password.toCharArray());
							

bills = wrap(db.getCollection("bill"), Bill.class, String.class);
dbCursor = bills.find().is("_id", billID);
if (dbCursor.hasNext()) { Bill result =  cursor.next();}
							

mongoimport --host --username --password --db --collection --file
							

HealthCheck

• Code

@Override
protected Result check() throws Exception {
    mongo.getDB(dbname).getCollectionNames();
    return Result.healthy();
}
							

{"MongoHealthCheck":{"healthy":true},"deadlocks":{"healthy":true}}
							

{"MongoHealthCheck":{"healthy":false,
	"message":"not authorized for query on tycoon-mongo.system.namespaces",
	"error":{"message":"not authorized for query on tycoon-mongo.system.namespaces",
	"stack":[...]
	"deadlocks":{"healthy":true}}
							

JSON POJOs

Use http://www.jsonschema2pojo.org/. Then apply simple annotations for Jackson and Hibernate Validatior.

@JsonInclude(JsonInclude.Include.NON_NULL)
public class Notification {

    @Id
    public String notificationID;

    @NotNull
    public String subscriptionID;

    @NotNull
    public String notificationURLForBillDataRetrival;

    @NotNull
    public String notificationURLForBillImageRetrival;

}
						

NOTE: Use @Id for UUID, not @ObjectId

Resources

Annotate opertion, metrics, url path and output format.
Jackson automatically parse POJO into specified mediaType.

@GET
@Timed
@Path("trigger/{subscriptionID}/{billID}")
@Produces(MediaType.APPLICATION_JSON)
public Notification triggerNotification(
	@PathParam("subscriptionID") String subscriptionID, 
	@PathParam("billID") String billID) {
   	...
    notification = new Notification();
    notification.notificationID = UUID.randomUUID().toString();
    notification.subscriptionID = subscriptionID;
    notification.notificationURLForBillDataRetrival = 
    	subscription.subscriptionCallBackURL + "?billID=" + billID;
    
    return notification;
}
						

Application Glue

Putting all of the above together

@Override
public void run(
	BillprotoConfiguration configuration, 
	Environment environment) throws Exception {

    MongoManaged mongoManaged = 
    	new MongoManaged(configuration.mongo);
    environment.lifecycle().manage(mongoManaged);
    environment.healthChecks().register(
    	"MongoHealthCheck", new MongoHealthCheck(mongoManaged));

    environment.jersey().register(new SubscriptionResource(mongoManaged));
    environment.jersey().register(new BillResource(mongoManaged));
}
						

Deploy

1. Install Java (defaults to OpenJDK)

sudo apt-get install java7-runtime-headless
						

2. Open firewall ports 8080 and 8081

gcutil addfirewall rest --description="http" --allowed="tcp:8080
gcutil addfirewall admin --description="Iadmin" --allowed="tcp:8081"
						

3. Copy file to server

gcutil --project={project-id} push {instance-name} {local-file} {remote-target-path}
						

4. Run fatjar

java -jar billproto-0.2-fat.jar server billproto.yml