Large-Scale

What we learned

Tri Tran @ DevDay DaNang

April 2019

about me

 

Tri D. Tran

Senior Technical Architect

tri.tran@nfq.asia

@moprit

 

NFQ Asia | 8Bit Rockstars
Member of .NFQ  Group

 

 

I'm happy if you call me 'Mụp'

we WILL discuss about

  • Elasticsearch, when and when NOT
     
  • What we learned after actualizing amazing ideas of Startup companies at NFQ Asia

we WON'T talk about

  • What's Elasticsearch
     
  • Elasticsearch and the others, which one is the best
     
  • DOTA 2, LoL, Counter Strike...

when?

1. "You Know, for Search"

  • Instant search
  • Spelling mistake
  • Relevant search result / Fuzzy search

when?

2. Analytics, document processing

  • Natural language processing
  • Statistical reports
  • Prediction
  • Percolation

when?

3. Centralized Log Management

4. Software as a Service (SaaS)

when NOT?

  • Not fluent
     
  • Economical issues
     
  • Transaction requirement
     
  • Just plain old lookup

Stay Safe

Lessons

Lesson 1

Indexing latency

There is no locking mechanism for the indexes.

Lesson 2

Bulk API

Refreshing is an expensive operation and that is why by default it’s made at a regular interval, instead of after each indexing operation

Reference

Lesson 3

Sharding

Primary Shard vs. Replica Shard

Shard Overallocation

Reference

Lesson 4

Index Aliases

Re-Indexing OR

Changing mapping with Zero Downtime

Lesson 5

Dedicated Master Nodes

Three dedicated master nodes, the recommended number, provides two backup nodes in the event of a master node failure and the necessary quorum to elect a new master.

 

The minimum_master_nodes setting is extremely important to the stability of your cluster. This setting helps prevent split brains, the existence of two masters in a single cluster.

Lesson n

Your problems?

Would you mind sharing them?

How did you solve?

we are HIRING

Simply send us an email with your enclosed updated CV to:

career@nfq.asia

Floor 11 Vincom Center
72 Le Thanh Ton, Ben Nghe Ward,
District 1, Ho Chi Minh City, Vietnam

 

54 Lam Hoanh, Phuoc My Ward,

Son Tra District, Da Nang City, Vietnam

Thanks

Tri D. Tran

tri.tran@nfq.asia

@moprit