elasticsearch
@ HootSuite
#elasticsearch?
- LogLevel debug
- search, not storage
es @ HootSuite
- 6 * m1.xlarge
- EBS storage, (usually) with p-iops
- haproxy LB frontend to kibana
#stats
~800 documents per second
~ 150 GB per day (70-200 range)
~ 400 GB per node
2 week rotation
#config
- pretty basic
- half available RAM
- 2000 IOPS EBS
#data
#shards #replicas
#issues
- cost
- EC2 and EBS performance unpredictable
- java heap size
- backup and restore
- logstash SPOF
- hot backup / S3
- split brain - startup timing
- network cost on master election
- EC2 DNS #lol
- tedious node addition
- tedious key removal
- metrics / graphite
#questions
Jeff Oliver :: HootSuite Ops
@jayo_bc
elasticsearch @ HootSuite
