Large-Scale

What we learned

Tri Tran @ DevDay DaNang

April 2019

about me

 

Tri D. Tran

Senior Technical Architect

tri.tran@nfq.asia

@moprit

 

NFQ Asia | 8Bit Rockstars
Member of .NFQ  Group

 

 

I'm happy if you call me 'Mụp'

we WILL discuss about

  • Elasticsearch, when and when NOT
     
  • What we learned after actualizing amazing ideas of Startup companies at NFQ Asia

we WON'T talk about

  • What's Elasticsearch
     
  • Elasticsearch and the others, which one is the best
     
  • DOTA 2, LoL, Counter Strike...

when?

1. "You Know, for Search"

  • Instant search
  • Spelling mistake
  • Relevant search result

when?

2. Analytics, document processing

  • Natural language processing
  • Statistical reports
  • Prediction
  • Percolation

when?

3. Centralized Log Management

4. Software as a Service (SaaS)

when NOT?

  • Not fluent
     
  • Economical issues
     
  • Transaction requirement
     
  • Just plain old lookup

Stay Safe

Lessons

Lesson 1

Indexing latency

There is no locking mechanism for the indexes.

Lesson 2

Bulk API

Refreshing is an expensive operation and that is why by default it’s made at a regular interval, instead of after each indexing operation

Reference

Lesson 3

Sharding

Primary Shard vs. Replica Shard

Shard Overallocation

Reference

Lesson 4

Index Aliases

Re-Indexing OR

Changing mapping with Zero Downtime

Lesson 5

Dedicated Master Nodes

Three dedicated master nodes, the recommended number, provides two backup nodes in the event of a master node failure and the necessary quorum to elect a new master.

 

The minimum_master_nodes setting is extremely important to the stability of your cluster. This setting helps prevent split brains, the existence of two masters in a single cluster.

Lesson n

Your problems?

Would you mind sharing them?

How did you solve?

we are HIRING

Simply send us an email with your enclosed updated CV to:

career@nfq.asia

Floor 11 Vincom Center
72 Le Thanh Ton, Ben Nghe Ward,
District 1, Ho Chi Minh City, Vietnam

 

54 Lam Hoanh, Phuoc My Ward,

Son Tra District, Da Nang City, Vietnam

Thanks

Tri D. Tran

tri.tran@nfq.asia

@moprit

Elasticsearch - What we learned

By Tri Tran

Elasticsearch - What we learned

Elasticsearch (ES) is not only a Search engine but also is a database that stores, and manages document-oriented and semi-structured data. There’s a bunch of reasons to use but a dozens of reason to avoid as well. I wants to share when we should and should not use it. Besides, I want to show lessons-learned and common mistakes in usage of ES such as Index Strategy, Data Mapping and so on. As a lead developer at NFQ Asia (8BIT Rockstars), his challenge is to build up many strong and confidence development teams. Together, we actualize amazing ideas of Startup companies at ASEAN.

  • 147

More from Tri Tran