Docs / Connections: Elasticsearch and OpenSearch

Set up Elasticsearch or OpenSearch

Verbiqo reads the monitoring and metadata APIs and never a document. It needs a role that can monitor, and refuses one that can write. Elasticsearch 7.17 and later, and OpenSearch 2.0 and later.

At a glance

Elasticsearch: the monitoring role

In Kibana (Stack Management › Roles) or Dev Tools, as an administrator:

PUT _security/role/verbiqo_monitor
{
  "cluster": ["monitor", "monitor_snapshot", "read_ilm", "read_slm"],
  "indices": [
    {
      "names": ["*"],
      "privileges": ["monitor", "view_index_metadata"]
    }
  ]
}

Then create an API key with that role (preferred over a password). Copy the encoded value from the result into API key on Verbiqo’s form:

POST _security/api_key
{
  "name": "verbiqo",
  "role_descriptors": {
    "verbiqo_monitor": {
      "cluster": ["monitor", "monitor_snapshot", "read_ilm", "read_slm"],
      "indices": [
        {
          "names": ["*"],
          "privileges": ["monitor", "view_index_metadata"]
        }
      ]
    }
  }
}

OpenSearch: the monitoring role

In OpenSearch Dashboards (Security › Roles) or as an administrator:

PUT _plugins/_security/api/roles/verbiqo_monitor
{
  "cluster_permissions": [
    "cluster_monitor",
    "cluster:admin/snapshot/get",
    "cluster:admin/repository/get",
    "cluster:admin/opendistro/ism/managedindex/explain",
    "cluster:admin/opendistro/ism/policy/get",
    "cluster:admin/opendistro/ism/policy/search",
    "cluster:admin/opensearch/snapshot_management/policy/get",
    "cluster:admin/opensearch/snapshot_management/policy/search",
    "cluster:admin/opensearch/snapshot_management/policy/explain",
    "cluster:admin/opensearch/insights/top_queries",
    "indices:admin/index_template/get",
    "indices:admin/template/get"
  ],
  "index_permissions": [
    {
      "index_patterns": ["*"],
      "allowed_actions": [
        "indices_monitor",
        "indices:admin/mappings/get",
        "indices:admin/get",
        "indices:admin/aliases/get",
        "indices:admin/data_stream/get",
        "indices:admin/resolve/index",
        "indices:admin/validate/query",
        "indices:data/read/field_caps"
      ]
    }
  ]
}

PUT _plugins/_security/api/rolesmapping/verbiqo_monitor
{
  "users": ["verbiqo_monitor"]
}

Create the user under Security › Internal users, or map the role to a backend role from your directory. On Amazon OpenSearch Service, map it to the user Verbiqo signs in as.

Add the connection

1. Open the form

   Extensions… › Elasticsearch (OpenSearch too).

2. Address and sign-in

   The URL of the HTTP endpoint (https://es.example.com:9200. On Elastic Cloud, the Elasticsearch endpoint). API key, or User and password.

3. Certificates

   A private CA: choose its certificate. A cluster that asks for a client certificate: add it and its key. Then Verification, in Elastic’s own terms: Full (a trusted CA and the host name, the default), Certificate (a trusted CA, any name: for connecting by IP address to a certificate issued for host names), or None (a lab only).

4. Test, then save

   Shows the product, version, health and TLS. An account that can write is refused here, with the privilege named.

Or from a terminal, the same connection, named logs:

verbiqo es add logs --url https://es.example.com:9200 --auth apikey --secret-stdin --ca /etc/ssl/company-ca.pem
verbiqo es test logs

Security logs (optional). Name log indices with --log-indices (or Log indices on the form) to let Verbiqo search them for security events and alerts with searches it builds itself: Security logs. Without it, no document is ever read.

The API key or password: --secret-stdin (piped in) or --secret-env VAR for scripts and CI. --auth basic --user U for a password. Client certificates: --cert client.pem --key client.key. Verification: --verification full|certificate|none (--no-verify is none). With an http:// URL nothing is encrypted, the API key or password included. The form says so and the test reports it.

Permission levels

Refused: superuser, any manage or write privilege on any index pattern, run_as, restricted system indices. On OpenSearch, which does not let an account read its own permissions: all_access, *_full_access and other writing roles, security REST API access, and admin certificates. A custom OpenSearch role is reported as not verifiable. Either way Verbiqo sends only reads from a fixed list.

Settings that help

Why it helps: none of these is required. Each lets Verbiqo see one more thing, so turn on only what you need.

Ask What should I enable? Verbiqo lists each item as ready or not, with the request that turns it on.

Optional additions to the role, each for one more thing Verbiqo can see:

Cross-cluster replication needs nothing more: the monitoring role reads its follower statistics.

Slow logs, per index or pattern:

PUT logs-*/_settings
{
  "index.search.slowlog.threshold.query.warn": "2s",
  "index.search.slowlog.threshold.query.info": "500ms",
  "index.indexing.slowlog.threshold.index.warn": "2s"
}

Query Insights (OpenSearch):

PUT _cluster/settings
{
  "persistent": {
    "search.insights.top_queries.latency.enabled": true,
    "search.insights.top_queries.latency.window_size": "10m",
    "search.insights.top_queries.latency.top_n_size": 20
  }
}

A nightly snapshot policy (Elasticsearch. OpenSearch uses POST _plugins/_sm/policies/nightly, which Verbiqo writes for you when it finds none):

PUT _slm/policy/nightly
{
  "schedule": "0 30 1 * * ?",
  "name": "<nightly-{now/d}>",
  "repository": "<repository>",
  "config": {
    "indices": ["*"]
  },
  "retention": {
    "expire_after": "30d",
    "min_count": 5,
    "max_count": 50
  }
}

Next: Ask Elasticsearch or OpenSearch.

Common errors

The test shows the message, and often a line under it saying what to change. Find the message here.