Documentation menu

Docs Connections: MySQL

MySQL checks

Every check Verbiqo can run on MySQL 8.0 and later (including Percona Server, Amazon RDS and Aurora), from a single server to InnoDB Cluster, grouped by area. You ask in plain words; the model says which areas the question needs, and Verbiqo runs those checks first. An unreachable primary or a lost majority is reported on any question.

How to read this page. Each section is one area. Why it helps says what you get from it. Ask is a question to type that runs it. The table under it lists what Verbiqo checks there: you do not run those one by one. When a check finds a problem, it appears under the answer as a finding, with its evidence and a suggested fix.

How to ask for each area

Attach the server with /mysql and the connection’s name, then ask.

AreaWords Verbiqo goes by without the modelAsk, for example
Right nowslow, hanging, stuck, blocked, locks, waiting, busy, timeouts, right now, currently, what is it doing, performanceThe orders app is hanging, what is going on?
A statement’s planpaste the statement into the question: SELECT, UPDATE, DELETE or INSERT … SELECTWhy is this slow? DELETE FROM orders WHERE …
Queries and indexesslowest, top queries, expensive, missing index, full scans, optimise, tune, indexWhich queries are slowest?
Replicationreplication, replica, slave, primary, secondary, lag, behind, GTID, binlog, binary log, failover, switchoverWhy is this replica behind?
Group Replication and InnoDB Clustercluster, group replication, InnoDB Cluster, router, quorum, split brain, high availability, HA, survive, redundancyCan the cluster survive a failover?
Every server in the topologytopology, all servers, every server, other servers, members, nodes, errantIs replication healthy across all servers?
Settings and stabilitytune, settings, configuration, variables, my.cnf, memory, buffer pool, innodb_*, redo, log buffer, thread cache, table cache, max_connections, stability, improve, recommendationsHow can I improve the stability of this server?
Schemas and tablesschema, tables, auto increment, primary keys, run out, MyISAM, engines, fragmented, free space, health checkAre any IDs about to run out?
The databases and their sizesdatabase sizes, how big, largest, list the databases, what databases; also read for any question about schemas, space or capacityFind all databases and their sizes (measured by Verbiqo from information_schema: data and index lengths per schema)
Securitysecurity, hardening, audit, root, accounts, users, privileges, grants, TLS, SSL, encryption, passwords, compliance, CIS, exposedIs this server secure?
Capacitycapacity, forecast, how long until, fill up, growth, sizing, scale up, headroom, planningCapacity planning for this server
What happenedwhat happened, timeline, incident, outage, yesterday, at 3pm, when did, root causeWhat happened on this server in the last hour?
Permission errorspaste the error: command denied, Access denied for user, errors 1142, 1143, 1370, 1044, 1045, 1227, 1419, 1449Our app gets this, can you check? ERROR 1142 (42000): …
What Verbiqo can seewhat should I enable, readiness, permissions, privilegesWhat should I enable?

Right now: sessions and locks

Why it helps: When something hangs now, it shows the sessions, locks and transactions holding others up, so you know what to clear.

Ask The orders app is hanging, what is going on?

Verbiqo samples sessions, lock waits and open transactions once a second for 10 seconds (see sampling).

CheckWhat it looks for
Row-lock blockingA session blocking others on row locks, the longest wait, and the session to end.
Metadata lockA DDL statement queued behind an open transaction, with every later statement on the table queued behind it, reported as one finding.
Idle in transactionA session idle inside an open transaction, holding locks and undo.
History listA long undo history list: purge cannot keep up, usually because of an old transaction. Past a million entries it is a finding; past ten million, critical.
Plans of running statementsStatements still running at the end of the sample, explained as they run (EXPLAIN FOR CONNECTION): a table read in full, a sort or a temporary table without an index. For another account’s session MySQL 8.4 allows this only to an account holding every global privilege; otherwise Verbiqo says the plans were not read.
DeadlocksDeadlocks during the sample.
Temporary tables on diskTemporary tables per second going to disk during the sample.
ConnectionsConnections near max_connections, and connections refused at the limit since the server started.

A statement’s plan

Why it helps: Explains why one statement is slow, or blocks others, from its plan, without running it, so you can fix that statement.

Ask Why is this slow? DELETE FROM orders WHERE created < …

Needs SELECT on the statement’s tables (Level 5). An UPDATE or DELETE is planned as the SELECT that finds its rows; a table named without its schema is given the one schema that has it.

CheckWhat it looks for
Full scanA large table read in full, saying whether an index exists that MySQL did not choose.
Function on an indexed columnA filter that wraps a column an index starts with (YEAR(created), LOWER(email)), so the index cannot be used, with the rewrite that compares the column itself.
Full index scanEvery entry of an index read, with nothing to start from.
Join without an indexA join matched in a join buffer instead of looked up.
Sorts and temporary tablesA GROUP BY, DISTINCT or ORDER BY not delivered in order by an index, over many rows.
Rows a write locksAn UPDATE or DELETE that reads a table in full locks every row it reads (InnoDB’s next-key locks), blocking other writers across the table.
Cascades, indexes and triggersForeign keys from other tables that cascade a delete or update, indexed or key columns an update changes, and triggers.

Queries and indexes

Why it helps: Shows the statements that cost the most and where indexes are missing, so you fix the few that matter.

Ask Which queries are slowest?

From the statement digests in performance_schema and sys. Verbiqo does not plan the statements in the digests. A statement you paste is planned when the account has SELECT on its tables (Level 5), and statements running during a live sample are explained as they run (above).

CheckWhat it looks for
One statement dominatesA single statement taking a large share of all statement time.
Slow statementA high average time per call, and when that time is really lock waiting.
Rows examinedA statement examining far more rows than it returns, with the index its filter needs, in the online form.
Full joinA join reading a table in full for every row of another.
Statement spillsA statement creating temporary tables on disk or sorting with merge passes.
Full scansTables with many rows read by full scans.
Redundant indexAn index covered by another.
Unused indexesIndexes not read, once the server has been up a week.

Replication

Why it helps: Shows each replica’s lag and errors and why, so you can trust reads from replicas and a failover.

Ask Why is this replica behind?

CheckWhat it looks for
Receiver stoppedA channel not receiving, with the error by cause: a purged binary log needs a re-seed.
Channels on a group secondaryOn a Group Replication secondary (single-primary mode), asynchronous channels such as an InnoDB ClusterSet’s are expected to be stopped: they run on the primary only, and starting one is refused (error 13117). Verbiqo says so, names the primary to check instead, and an answer may not advise starting replication on that member.
Applier errorA channel that stopped applying. A duplicate key is a divergence, not something to skip.
LagHow far behind, from the source’s commit times, allowing for a configured delay.
No auto-positionA channel using file positions although GTIDs are on.
Writable replicaA replica with super_read_only off.
Relay log recoveryrelay_log_recovery off: a replica crash can corrupt its relay log.
Source durabilitysync_binlog or innodb_flush_log_at_trx_commit not 1 on a source.
Binary log retentionBinary logs kept too briefly for a replica to catch up after an outage.
Binary log offNo binary log: no replicas and no point-in-time recovery.
Semi-sync fell backSemi-synchronous replication enabled but not active.

Group Replication and InnoDB Cluster

Why it helps: Shows the group’s members, roles and queues, so you know whether it survives losing a member.

Ask Can the cluster survive a failover?

CheckWhat it looks for
Majority lostMembers unreachable, so the group cannot accept writes.
Primary unreachableNo member accepts writes.
Member stateA member recovering, in error or offline.
No fault toleranceToo few members online to survive losing one.
Member behindA member with transactions waiting to apply or certify.
Mixed versionsMembers running different versions.
ConflictsMulti-primary transactions rolled back for changing the same rows.
Auto-rejoin offA member expelled by a brief network problem will not rejoin by itself.
Metadata and group disagreeA member in the group but not in InnoDB Cluster’s metadata, or the reverse.
MySQL RouterNo Router registered for the cluster, or a Router older than the servers it routes to.

Every server in the topology

Why it helps: Shows the other servers as the attached one sees them (members, replicas, lag), so one question gives the state of the whole topology, and says which servers were not read themselves.

Ask Is replication healthy across all servers?

Asked about the whole topology, Verbiqo reports the other servers as the attached one sees them (group members and their state, replicas and their lag) and names each as not read: a question reaches only what is attached, so no saved connection is used that you did not attach. To read another member, attach it in a conversation of its own. If the question itself asks for it, for example read the other members with this connection’s credentials, Verbiqo reaches the servers the attached one reports, at the addresses it gives and with the same credentials; a saved connection is still never used.

CheckWhat it looks for
Errant transactionsA replica with transactions its source does not have: they break the next failover.
Duplicate server_idServers sharing a server_id.
Replica older than sourceA replica on an older version than its source.
Failover settings differA server that would take over from the primary with different settings.

Settings and stability

Why it helps: Finds settings and conditions that make the server unstable or slow, with the change for each.

Ask How can I improve the stability of this server?

CheckWhat it looks for
Buffer pool missesThe share of InnoDB page reads going to disk, sized against the machine’s memory when a metrics source has it.
Buffer pool against memoryThe buffer pool against the machine’s memory (about 60%, half on a machine of 4 GB or less), before any read goes to disk; and a pool so large the machine would swap. The memory comes from a metrics source, or from the question: 16 GB of memory, 4 cores, SSD.
SSD and network storageinnodb_io_capacity and innodb_flush_neighbors still at the spinning-disk defaults of older versions.
Memory the server could takeWhat is allocated at start plus each connection’s buffers at the real peak (a warning when that exceeds the machine), and the worst case with the temporary-table cap and max_connections, naming the setting to cap.
After a changeA setting changed since the server started is not pushed further on counters that began before the change (variables_info says when, and by whom); a statement for a value already set is not offered.
Waits for free pagesQueries waiting for a free buffer pool page.
Redo capacityThe redo log against an hour of writes.
Log bufferWrites waiting for the log buffer.
Thread cacheConnections needing a new thread.
Table cacheThe table cache overflowing.
Binary log cacheTransactions spilling their binary log cache to disk.
Temporary tables on diskThe share of internal temporary tables going to disk.
max_connections far too highA limit far above the most ever used.

Schemas and tables

Why it helps: Finds IDs about to run out and table design problems before inserts fail.

Ask Are any IDs about to run out?

CheckWhat it looks for
Auto-increment running outKeys near the end of their type.
No primary keyTables without one: they stall replicas and are refused by Group Replication.
PartitionsA RANGE-partitioned table whose last bound is already in the past or within two weeks, and rows piling up in a MAXVALUE catch-all partition.
Not InnoDBTables on another engine.
FragmentedSpace held by deleted rows.

Security

Why it helps: Shows accounts with too much access, weak authentication and unencrypted connections, so you close the gaps an audit would find.

Ask Is this server secure?

CheckWhat it looks for
EncryptionWhether encryption is required, and which connections arrive without it.
Remote rootroot connecting from other hosts.
Deprecated authenticationmysql_native_password active.
Password validationNothing checking the strength of new passwords.
AuditNo audit log.
File accesslocal_infile on, or secure_file_priv empty.

The same checks are controls in the security review, which covers every saved connection at once and maps each to CIS Controls, ISO 27001 and SOC 2.

Permission errors

Why it helps: Turns a pasted MySQL permission error into the grant that should fix it, for the right account and schema, and no wider.

Ask Our app gets this, can you check? ERROR 1142 (42000): SELECT command denied to user …

ErrorWhat Verbiqo gives
1142, 1143: table or column refusedThe GRANT on that table or column, in the schema that holds it (asked if several do).
1370: routine refusedThe GRANT EXECUTE on the routine.
1044: database refusedA GRANT on the database, with the privileges for you to choose.
1045: login refusedNot a privilege: the password, the host the account allows, or the account itself.
1227: administrative privilegeWhich one, and the statement to change instead (a DEFINER or SET in a restored dump); no grant.
1419: function refused with binary logging onDeclare it DETERMINISTIC or READS SQL DATA, or, only if every routine creator is trusted, the server setting that allows it.
1449: definer missingWhere the dropped account is the definer, and the choice: recreate it, or have the object run as the caller or another account.

More: Permission errors.

Ready-made reports

A report is one fixed reading of one area: no question, no model, the same output every time. Use one to look at an area yourself, or to share what it shows.

From a terminal, it prints the report:

verbiqo mysql run shop lock-waits

In verbiqo chat, it loads the report into the conversation so you can ask about it. help lists the reports:

/mysql shop lock-waits
/mysql shop help

In the desktop apps, there is no report command: ask in your own words, and Verbiqo reads what the question needs.

ReportShows
sessionsSessions doing something now, longest first.
long-transactionsInnoDB transactions open longest, and what they hold.
lock-waitsRow lock waits: who waits, who blocks, for how long.
metadata-locksMetadata lock waits (DDL waiting on an open transaction).
replicationReplication channels: receiver, applier, errors and lag.
group-replicationGroup Replication and InnoDB Cluster members, roles and queues.
top-statementsStatements by total time.
full-scansStatements that read tables in full without an index.
temp-tablesStatements that create temporary tables on disk.
unused-indexesIndexes with no reads since the server started.
errorsErrors the server has raised, most frequent first.
connectionsConnections by account and host, against the limit.
settings-of-noteSettings changed from their compiled defaults, and when.

The account’s permission levels are in Set up MySQL. Capacity checks for MySQL are on Capacity checks, and what Verbiqo finds in the MySQL error and slow logs is on Log findings.

Something missing or unclear? Email hello@verbiqo.app.