Python Developer'lar için Elastisearch​

Bahattin ÇİNİÇ

Software Developer @adphorus

http://bahattincinic.com

​http://github.com/bahattincinic

Elasticsearch Nedir ?

  • Java ile geliştirildi (cross-platform)

  • Open Source

  • full-text search & analytics engine

  • Lucene Tabanlı

  • RESTful

  • Scalable & Distributed

  • Near Real Time

Elastic Ürünleri

  • Elasticsearch

  • Kibana

  • Logstash

  • Beats

  • X-Pack

  • Cloud
     

(2012 Yılında Kuruldu)

Kullanım Alanları

  • E-Ticaret sitelerinde search & filter

  • Loglama, monitoring, alert

  • Foursquare & Zomato gibi sitelerde gis desteği sayesinde konum bazlı mekan arayabilmek

  • Haber sitelerinde haber içeriğine göre arama yapmak ve renklendirmek.

  • Online rezervasyon sitelerinde Şehir isimlerine göre autocomplete yapabilmek.

  • Crawling & Document Processing

Özellikleri

  • Cluster, Shard , Replication

  • Near Realtime

  •  Node & Index & Doc Type

  • full-stext search, boost, filter, aggregation

  • Geographic information system (gis)

  • RESTful

  • Apache ürünlerine destek (spark, storm, pig, hive, hadoop). Apache lisansına sahip.

  • TTL (Time to live)

  • facetch, autocomplete (suggestion)

  • yaml configuation (elasticsearch.yml)

Documents, Types, İndexes, Segment

Elasticsearch RDBMS
Index Database
Shard Shard
Mapping Table
Field Field
JSON Object Tuple


PostgreSQL => Databases => Tables => Columns/Rows
Elasticsearch => Indices => Types => Documents with Properties

Cluster, Node, Shard

http://localhost:9200

Query Sorgulamarı

Yeni index oluşturmak

$ curl -XPUT 'localhost:9200/places?pretty' -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "index" : {
            "number_of_shards" : 5, 
            "number_of_replicas" : 1 
        }
    }
}
'

Query Sorgulamarı

Mapping Güncellemek

curl -XPUT 'localhost:9200/places/_mapping/places?pretty' -H 'Content-Type: application/json' -d'
{
    "properties": {
        "title": {
          "type": "string"
        },
        "point": {
          "type": "geo_point"
        },
        "id": {
          "type": "integer"
        },
        "category": {
          "type": "string"
        }
    }
}
'

Query Sorgulamarı

Veri Eklemek (Document)

curl -XPUT 'http://localhost:9200/places/places/1?pretty' -H 'Content-Type: application/json' -d'
{
    "id" : 2,
    "point": "41.12,-71.34",
    "title": "Hipo",
    "category": "Ofis"
}
'

Query Sorgulamarı

Eklediğimiz Verinin Detayına Gitmek (Retrieve)

curl -XGET 'http://localhost:9200/places/places/1'
{
   "_index":"mekanlar",
   "_type":"mekanlar",
   "_id":"1",
   "_version":1,
   "found":true,
   "_source":{
      "id":1,
      "point":"41.12,-71.34",
      "title":"Hipo",
      "category":"Ofis"
   }
}

Query Sorgulamarı

Eklediğimiz tüm verileri Çekmek

curl -XGET 'http://localhost:9200/places/places/_search'
    -H 'Content-Type: application/json' -d'
{
    "query": {
        "match_all": {}
    }
}
'

{
   "took":1,
   "timed_out":false,
   "_shards":{
      "total":1,
      "successful":5,
      "failed":0
   },
   "hits":{
      "total":1,
      "max_score":1.0,
      "hits":[
         {
            "_index":"mekanlar",
            "_type":"mekanlar",
            "_id":"1",
            "_score":1.0,
            "_source":{
               "id":1,
               "point":"41.12,-71.34",
               "title":"Hipo",
               "category":"Ofis"
            }
         },
      ]
   }
}

Query Sorgulamarı

Eklediğimiz Verileri Filtrelemek

curl -XGET 'http://localhost:9200/places/places/_search'
    -H 'Content-Type: application/json' -d'
{
    "query": {
        "term": { "category": "Ofis" }
    }
}
'

{
   "took":1,
   "timed_out":false,
   "_shards":{
      "total":1,
      "successful":5,
      "failed":0
   },
   "hits":{
      "total":1,
      "max_score":1.0,
      "hits":[
         {
            "_index":"mekanlar",
            "_type":"mekanlar",
            "_id":"1",
            "_score":1.0,
            "_source":{
               "id":1,
               "point":"41.12,-71.34",
               "title":"Hipo",
               "category":"Ofis"
            }
         },
      ]
   }
}

Python Kütüphaneleri

  • elasticsearch (​official client)
  • elasticsearch-dsl-py (official client)
  • django-haystack (Sadece Django)

Client kullanmak yerine zaten RESTful oldugu icin direk requests paketini kullanarak da elasticsearch u kullanabilirsiniz

Elasticsearch Güvenliği

- Kendi içerisinde Auth. mekanizması barındırmıyor

- Nginx arkasına koyulabilir

- Public bir yerden erişimi yoksa private ip ile erişim sağlanabilir

Deployment

  • Docker
    (https://hub.docker.com/_/elasticsearch)
  • Google cloud (https://www.elastic.co/about/partners/google-cloud-platform)
  • AWS
    (https://aws.amazon.com/elasticsearch-service)
  • Elasticsearch Cloud
    (https://www.elastic.co/cloud)
  • Ansible
    (https://github.com/elastic/ansible-elasticsearch)

Plugins

  • https://chrome.google.com/webstore/detail/sense-beta/lhjgkmllcaadmopgmanpapmpjgmfcfig?hl=en (Elasticsearch Developer Console)
  • https://github.com/mobz/elasticsearch-head (web front end for an elastic search cluster)
  • https://github.com/polyfractal/elasticsearch-inquisitor (debug queries)

Demo

https://github.com/bahattincinic/elasticsearch_demo

Teşekkürler

Made with Slides.com