Large-Scale
What we learned
Tri Tran @ DevDay DaNang
April 2019
about me
Tri D. Tran
Senior Technical Architect
tri.tran@nfq.asia
@moprit
NFQ Asia | 8Bit Rockstars
Member of .NFQ Group
I'm happy if you call me 'Mụp'
we WILL discuss about
- Elasticsearch, when and when NOT
- What we learned after actualizing amazing ideas of Startup companies at NFQ Asia
we WON'T talk about
-
What's Elasticsearch
-
Elasticsearch and the others, which one is the best
- DOTA 2, LoL, Counter Strike...
when?
1. "You Know, for Search"
- Instant search
- Spelling mistake
- Relevant search result / Fuzzy search
when?
2. Analytics, document processing
- Natural language processing
- Statistical reports
- Prediction
- Percolation
when?
3. Centralized Log Management
4. Software as a Service (SaaS)
when NOT?
- Not fluent
-
Economical issues
-
Transaction requirement
- Just plain old lookup
Stay Safe
Lessons
Lesson 1
Indexing latency
There is no locking mechanism for the indexes.
Lesson 2
Bulk API
Refreshing is an expensive operation and that is why by default it’s made at a regular interval, instead of after each indexing operation
Reference
Lesson 3
Sharding
Primary Shard vs. Replica Shard
Shard Overallocation
Reference
Lesson 4
Index Aliases
Re-Indexing OR
Changing mapping with Zero Downtime
Lesson 5
Dedicated Master Nodes
Three dedicated master nodes, the recommended number, provides two backup nodes in the event of a master node failure and the necessary quorum to elect a new master.
The minimum_master_nodes setting is extremely important to the stability of your cluster. This setting helps prevent split brains, the existence of two masters in a single cluster.
Lesson n
Your problems?
Would you mind sharing them?
How did you solve?
we are HIRING
Simply send us an email with your enclosed updated CV to:
career@nfq.asia
Floor 11 Vincom Center
72 Le Thanh Ton, Ben Nghe Ward,
District 1, Ho Chi Minh City, Vietnam
54 Lam Hoanh, Phuoc My Ward,
Son Tra District, Da Nang City, Vietnam
Thanks
Tri D. Tran
tri.tran@nfq.asia
@moprit