Set up Kubernetes

Verbiqo reads a cluster through its API server, with an account that can list and get and nothing else. It never reads Secrets or ConfigMaps, never execs into a pod, and refuses to connect with an account that can change the cluster.

At a glance

Supported tasks	Find why pods crash or wait, what a rollout changed, whether there is room, what breaks in a drain or upgrade, and how secure the cluster is. Ask Kubernetes and Kubernetes checks.
Limitations	Secrets and ConfigMaps are never read, so a value held in one is never seen. Kubernetes keeps events for about an hour. Older history comes from the objects’ own times, or from Prometheus and logs.
Supported versions	Kubernetes 1.23 and later, where every resource Verbiqo reads is served. Tested on 1.36. An older cluster still connects, and what it does not serve is listed as not read: autoscalers below 1.23, and CronJobs, disruption budgets and EndpointSlices below 1.21. The account’s own name is read on 1.27 and later. EKS, GKE, AKS, OpenShift and k3s are recognised.
Authentication	A kubeconfig context, using its own sign-in: a token, a client certificate, or an exec plugin. An exec plugin is a program that fetches credentials, such as `aws eks get-token`, `gke-gcloud-auth-plugin`, `kubelogin` or `oc`. Or a service-account token with the cluster’s CA certificate.
Minimum permissions	A ClusterRole that can only `get` and `list`, bound to a service account. `pods/log` is optional. To keep to some namespaces, bind it with a RoleBinding in each. Commands: Create the read-only account and What the role allows.
Connection test	Press Test Connection in Extensions… › Kubernetes, or run `verbiqo k8s test <name>`. Success shows Connected., the platform, version and node count, and the account. It then shows what the account reads: the whole cluster or some namespaces, live use (metrics-server), and whether logs are allowed.
First useful question	Attach with `/k8s <name>`, then ask Why is checkout crashing? Expect crash loops, OOM kills (out-of-memory restarts), image pulls, missing Secrets or ConfigMaps, and pending pods with the scheduler’s reason. With logs on, the crashed run’s log is read too.
Safety controls	Only GET and LIST of named resources. Nothing is watched or followed, and nothing is written. Secrets and ConfigMaps are never read. `exec`, `attach`, `portforward`, `proxy` and service-account `token` are never sent. Before reading, Verbiqo asks the API server whether the account can exec, create, delete, change or scale workloads, bind roles, impersonate or mint tokens. If it can, the connection is refused. An administrator’s kubeconfig is refused this way. Logs are read only when the connection allows them, and each line is masked as it is read. Credentials in environment values, commands and annotations are masked too.
Operational limits	20 seconds per request. Lists are read 500 objects a page, at most 5,000 of a kind (3,000 events and ReplicaSets). Three requests at a time. The model asks for up to eight reads per question. Logs: at most four containers, 120 lines each. Your organization can lower these under `k8s`: Protect your systems.

1. Create the read-only account

As a cluster administrator, save this as verbiqo-reader.yaml: a ClusterRole that can only get and list, a service account, and the binding between them.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: verbiqo-reader
rules:
- apiGroups: [""]
  resources: [pods, services, endpoints, nodes, namespaces, events, persistentvolumeclaims, persistentvolumes,
              replicationcontrollers, resourcequotas, limitranges, serviceaccounts]
  verbs: [get, list]
- apiGroups: [""]
  resources: [pods/log]
  verbs: [get]
- apiGroups: [apps]
  resources: [deployments, replicasets, statefulsets, daemonsets]
  verbs: [get, list]
- apiGroups: [batch]
  resources: [jobs, cronjobs]
  verbs: [get, list]
- apiGroups: [autoscaling]
  resources: [horizontalpodautoscalers]
  verbs: [get, list]
- apiGroups: [policy]
  resources: [poddisruptionbudgets]
  verbs: [get, list]
- apiGroups: [networking.k8s.io]
  resources: [ingresses, ingressclasses, networkpolicies]
  verbs: [get, list]
- apiGroups: [discovery.k8s.io]
  resources: [endpointslices]
  verbs: [get, list]
- apiGroups: [rbac.authorization.k8s.io]
  resources: [roles, rolebindings, clusterroles, clusterrolebindings]
  verbs: [get, list]
- apiGroups: [storage.k8s.io]
  resources: [storageclasses, volumeattachments]
  verbs: [get, list]
- apiGroups: [scheduling.k8s.io]
  resources: [priorityclasses]
  verbs: [get, list]
- apiGroups: [metrics.k8s.io]
  resources: [pods, nodes]
  verbs: [get, list]
- apiGroups: [networking.istio.io, security.istio.io, gateway.networking.k8s.io, policy.linkerd.io]
  resources: [virtualservices, destinationrules, gateways, serviceentries, sidecars, peerauthentications, authorizationpolicies,
              httproutes, grpcroutes, gatewayclasses, servers, serverauthorizations]
  verbs: [get, list]
- nonResourceURLs: [/version, /metrics, /readyz, /livez]
  verbs: [get]
---
apiVersion: v1
kind: Namespace
metadata:
  name: verbiqo
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: verbiqo-reader
  namespace: verbiqo
automountServiceAccountToken: false
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: verbiqo-reader
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: verbiqo-reader
subjects:
- kind: ServiceAccount
  name: verbiqo-reader
  namespace: verbiqo

kubectl apply -f verbiqo-reader.yaml

The same YAML, always matching your version of Verbiqo:

verbiqo k8s readiness

To keep Verbiqo to some namespaces, bind the same ClusterRole with a RoleBinding in each instead of the ClusterRoleBinding:

verbiqo k8s readiness --namespaces shop,payments

What the role allows

Part	Allows
Pods, services, nodes, namespaces, events, claims and volumes, plus Deployments, ReplicaSets, StatefulSets, DaemonSets, Jobs, CronJobs	What is running, how it is doing, and what changed.	Required
Autoscalers, disruption budgets, quotas, limit ranges, ingresses, endpoint slices, storage classes	Capacity, rollouts, drains and traffic.	Recommended
Roles, bindings, network policies, service accounts	The security review: who is cluster-admin, which namespaces are open.	Recommended
`metrics.k8s.io` pods and nodes	Live CPU and memory from metrics-server.	Recommended
Istio, Gateway API and Linkerd objects	Routes, subsets, mTLS and authorization policies, gateways. Read only where installed. None of them holds a credential.	Recommended with a mesh
`pods/log`	The last lines of a crashing container. Used only if the connection allows logs.	Optional
`/metrics`	The API server’s count of deprecated APIs still called, for upgrades.	Optional

No Secrets, no ConfigMaps (they often hold passwords), no watch, no write verb, no wildcard. Before reading anything Verbiqo asks the API server whether its account can exec, create, delete, change or scale workloads, bind roles, impersonate or mint tokens. If it can, Verbiqo refuses the connection and says why.

2. Add the connection

In the app: Extensions… › Kubernetes › Add, choose a context or enter a server and token, then Test Connection.

A kubeconfig context uses the kubeconfig’s own sign-in, exec plugins included (aws eks get-token, gke-gcloud-auth-plugin, kubelogin, oc). Verbiqo copies nothing out of the file.

verbiqo k8s contexts
verbiqo k8s add prod --context prod-eks
verbiqo k8s test prod

The default kubeconfig is used. Another file: --kubeconfig FILE on contexts and add. If the plugin needs you to sign in (a browser or a code), run kubectl get ns with that context once first: Verbiqo never waits at a prompt.

A service-account token for the account above, kept in the Keychain or Credential Manager:

kubectl -n verbiqo create token verbiqo-reader --duration=8760h | verbiqo k8s add prod --server https://api.prod.example:6443 --ca cluster-ca.crt --token-stdin

In a script, --token-env VAR reads the token from a variable instead of standard input.

Options for either:

Option	Does
`--namespaces shop,payments`	Reads only these namespaces.
`--logs`	Lets Verbiqo read the last lines of failing containers, masked as they are read. The matching role without `pods/log`: `verbiqo k8s readiness --no-logs`.

TLS and certificates

Connection	How the API server is checked
A kubeconfig context	As kubectl would: the cluster’s `certificate-authority` or `certificate-authority-data`, its `tls-server-name` if set, and `insecure-skip-tls-verify` if the file says so. Nothing to add.
A token connection	`--ca FILE`, the cluster’s CA certificate (what the kubeconfig holds as `certificate-authority`). `--verification full` (the default) checks the certificate and the host name. `certificate` checks the certificate only, for connecting by IP address to a certificate issued for a name. `none` checks nothing, for a lab.

The security review also reads the API server’s certificate itself: issuer, expiry, names, key size and protocol version.

Settings that help

metrics-server: live CPU and memory. Most managed clusters have it. Otherwise kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml.
A Prometheus connection with cAdvisor and kube-state-metrics: a day of CPU throttling and peak memory against limits, not only the moment.
Splunk or Elasticsearch with the container logs: events and logs older than the hour Kubernetes keeps them.

Ask what should I enable? with the cluster attached to see each as ready or not.

Load on the API server

A question reads only the kinds its areas need, in the namespaces it is about when it names something: 500 objects a page, at most 5,000 of a kind (3,000 events and replica sets), three requests at a time, with every check worked out from that one read. Nothing is watched. Logs are read for at most four failing containers, 120 lines each. Your organization’s extension limits apply under the name k8s.

The names map

To find what a question is about (checkout, payments), Verbiqo keeps a map of the cluster’s names: namespaces, Deployments, StatefulSets, DaemonSets, CronJobs and Services, read as metadata only, never specs or status. It is written as one file per connection under k8s-maps/ in the folder that holds the connections (~/Library/Application Support/Verbiqo/extensions/ on macOS, %LOCALAPPDATA%\Verbiqo\extensions\ on Windows, ~/.config/verbiqo/extensions/ on Linux), readable by its owner only, and reused for ten minutes before it is read again (VERBIQO_K8S_MAP_TTL, in seconds, up to a day. 0 reads it on every question).

Security and the record of reads

The security review checks the cluster itself (anonymous access, Pod Security, cluster-admins, wildcard roles, credentials in manifests, a supported version) with the same account, and every request Verbiqo sends is on the access record.

Next: Ask Kubernetes.

Common errors

The test shows the message, and often a line under it saying what to change. Find the message here.

Message or symptom	Likely cause	What to do	Needs an administrator?
… can … on this cluster. Verbiqo only reads, and connects only with an account that can do nothing else.	The context uses an administrator’s account, or one that can change the cluster.	Create the read-only account with `verbiqo k8s readiness`, then connect with it.	Yes: creating the role and binding needs a cluster administrator.
The API server did not accept the credentials (401): the token may have expired.	The cloud sign-in or the token has expired.	Sign in again (for example `aws sso login`, `gcloud auth login` or `az login`), or give the connection a fresh token.	Only if an administrator issued the token.
Signing in with “…” failed, with If it needs you to sign in (a browser or a code)…	The exec plugin wants you to sign in, and Verbiqo never waits at a prompt.	Run `kubectl get ns` with that context once in a terminal, then test again.	No.
The kubeconfig signs in with “…”, which is not installed or not on the PATH.	The exec plugin program is missing on this computer.	Install it, or give its full path in the kubeconfig.	No.
… signs in with the old “…” auth provider, which kubectl itself no longer supports.	An old kubeconfig from before exec plugins.	GKE: install `gke-gcloud-auth-plugin` and run `gcloud container clusters get-credentials` again. AKS: run `kubelogin convert-kubeconfig`.	No.
Could not reach … with The API server’s certificate is not trusted	A token connection without the cluster’s CA certificate.	Give the connection the cluster’s CA file (`--ca`). By IP address, use verification `certificate`.	Only to obtain the CA file.
… did not answer: a private cluster may need a VPN or bastion.	The API server is on a private network.	Connect the VPN or bastion (a jump host) the cluster needs, then test again.	Only if you lack VPN access.
This connection does not allow reading logs.	Logs are off for this connection.	Turn on Read pod logs for it (`--logs` in a terminal). The role also needs `pods/log`.	Only if the role lacks `pods/log`.