Technology Stack

Development

• Nodejs/Expressjs for web backend CRUD, API endpoints
• Angularjs,js, bootstrap, css front end
• Java(for all long running processes like scrapping, caching)
• JSON web Token(jwt) for API authentication
• MongoDB - database
• Redis - data caching
• Kafka - publish subscribe
• RabbitMQ/Redis - Message Queue
• Nginx - Load Balancer

Technology Stack (contd.)

Deployment

• AWS/Digitalocean(PAAS)
• Jenkins - Continious Integration
• Ansible - configuration management
• Nagios - monitoring
• Docker - system images
• Terraform - Automation

Key Data Models

User Details(MongoDB Collection)

Name 
Email 
Phone
Social : {Facebook,twitter etc}
Userid(UUID(universally unique identifier))
LastUpdatedDate
Status:{active|inactive}

Key Data Models(contd.)

Customer Engagements(MongoDB Collection)

UserId:{ref userdetails documents}
web: [{view,view_desc,timestamp}]
call:[{timestamp,call duration}]
SMS:{
    to:[{timestamp,payload}]
    from:[{timestamp,payload}]
}
Whattsapp: {
    to:[{payload_type:{text|image|file},payload,timestamp}]
    from:[{payload_type:{text|image|file},payload,timestamp}]
}

Key Data Sets(contd.)

Customer Engagement(Redis HASH)

userid:[{engagement,timestamp,comments}]

key: userid, value:JSON object of customer engagement data

*Preprocess and cache the customer engagements in Redis HASH structure

*Subscribe to Kafka channel "cmd:userengagementupdated"

Key Data Sets(contd.)

*Cached Inverted indexes to support faster search results 

* Redis SET, ZSET structures : name, email, phone number, search text etc.

eg;

ph_81226418:useridx

ph_81226419:useridy

nm_abc:useridx,useridy

nm_pqr:useridp,useridq,useridr

*Subscribe to Kafka channels such as "cmd:user added", "cmd:userdeleted" and update the index

How to get customer engagement data?

SMS?
- Android APP, write a braodcast receiver to capture SMS received event. Read, parse the SMS and post it to server.

Call?
- Android APP, write a broadcast receiver to capture incoming/outgoing call event.Read data, post it to server.

Email data?

Possible to scrap if we have our own email server.

Website Page views?

API sample :

http://somehost.com/category/objectid  --> object accessed by user
http://somehost.com/category    ---> Category acessed by user

Store the API data acessed by user from App(Web/mobile/desktop)

*Store data in Customer engagements collection

How to browse the history of all communications with the customer on desktop and mobile, by searching up the name, email, phone number or other identifying attributes?

Redis inverted index would support various search attributes such as name, email, phone number etc. to browse the history of all communications.

BONUS: along with communication, I want to see the page views of the customer on Carro.sg

Log every API accessed by user and store it in customer engagement collection in mongodb which is ultimately cached into corresponding Redis customer engagement

HASH for faster access.

BONUS: when the customer calls on my smart phone, I want to immediately see the customer history popup.

Android Broadcast receiver would read phone number and send a request to API server to fetch the data. 
Use the same search functionality as mentioned in previous slide

Same user can come through multiple channels, how to prevent duplication?

Authorisation is must to allow customer to interact with system. Possible channels using which customer can interact with system are : 

1. Call (auth phone number)

2. SMS(auth phone number)

3. Whattsapp(auth phone number)

4. email(auth emailid)

5. Facebook(auth emailid, get the user data(email) from FB api)

User Details mongodb collection stores phone number, email of customer and uses same UUID to distinguish return customer. If the email or phone number are found then link the access accounts.

Thank you!