python 3.x - Get indices from elastic search server

Question

I have installed elastic search through docker-compose on a machine. The version of elastic search in the yaml file is 1.11.0 and pyelasticsearch 7.6.0. I am trying to get all the indices from the elastic search server. The connection to the server is successful, I can see the message "Connection to ES Server successful", but I cannot get the indices. the line "es.indices.get_alias("*")" fails

try:
    es = Elasticsearch("https://admin:admin@localhost:9200",
                       use_ssl = False,
                       ca_certs=False,
                       verify_certs=False)

    print("Connection to ES Server successful")
except:
    print("Unable to connect to server")
    exit(1)

# get all wazuh indexes
lsindex = []
for i in es.indices.get_alias("*"):
    if i.startswith(ALERTS_PREFIX):
        lsindex.append(i)

File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
    raise err
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 246, in perform_request
    method, url, body, retries=Retry(False), headers=request_headers, **kw
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 344, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 301, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 168, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f6255479110>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./mcode.py", line 185, in <module>
    aut_for_tool()
  File "./mcode.py", line 78, in aut_for_tool
    for i in es.indices.get("*"):
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/indices.py", line 193, in get
    "GET", _make_path(index), params=params, headers=headers
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/transport.py", line 390, in perform_request
    raise e
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/transport.py", line 365, in perform_request
    timeout=timeout,
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 258, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6255479110>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6255479110>: Failed to establish a new connection: [Errno 111] Connection refused)

question from:https://stackoverflow.com/questions/65842483/get-indices-from-elastic-search-server

Answer 1

This is my code example:

#!/usr/bin/env python
import sys

try:
    from elasticsearch import Elasticsearch
except Exception as e:
    print("No module 'elasticsearch' found.")
    sys.exit()


try:
    es = Elasticsearch("https://admin:admin@localhost:9200",
                       use_ssl = True,
                       ca_certs=True,
                       verify_certs=False)

    print("Connection to ES Server successful")
except:
    print("Unable to connect to server")
    exit(1)

# get all wazuh indexes
lsindex = []
for i in es.indices.get("*"): 
    print(str(i))

And it produces the following output:

# python3 test.py

.opendistro_security
security-auditlog-2021.02.19
wazuh-alerts-4.x-2021.02.19

Can I ask why are you getting all Wazuh indices in your script? Understanding your use case could help me to point you to a better solution. If it's something related to reporting, Opendistro for elasticsearch has a module for it.

Regards, Alberto R