Who asked, when, which model answered, what it cost, and what shape the answer had. The fields that turn an AI log into evidence, the two kinds of row worth keeping apart, and the case for leaving conversation content where it was written.
Updated · 7 min read
An auditor arriving at an AI feature is asking one thing: can this organisation account for what it did. How clever the model is comes second. Five questions carry almost all of the answer.
A log that answers those five can be handed to a reviewer. A log that answers three starts a conversation about the other two, usually at the worst possible moment.
The read steps are the field most logs leave out, and the field a reviewer reaches for first. An answer becomes checkable once you can see what it consulted. Otherwise a reviewer is comparing a paragraph against their own memory of the data. That is the position the log was built to get them out of.
Usage from a conversational assistant and usage from a feature inside a tool are different events. They are recorded as different kinds of row.
Folding the two together would give one table with half its columns permanently empty. An empty column reads as a missing value rather than a category that never had one. Two honest shapes beat one shape with holes in it.
The strongest thing an AI log can do is record everything about a call except its content. By default these rows carry metadata alone. The reasons are about governance, not storage.
A default row records who, where, which model, how many tokens, what was read, and how many actions were proposed.
Full capture stays available, and it is switched on deliberately. It is there for a regulated deployment whose own policy requires content audit and says so in writing, and it stays the exception. A team that needs it knows it needs it, and everybody else keeps a log they can retain without a second review.
Where the assistant runs a query, the query itself is audited. The statement it ran, how long it took, how many rows came back, and whether it succeeded. The returned cells stay out of the record. The row count is the shape worth keeping, and the data is already where it belongs.
Refusals are recorded with the same weight as successes, and this is the part worth arguing for. A query the guard turned away is the most informative row in the file. It is the one place a reader can watch the boundary do its job. A log that keeps only the calls that worked has quietly deleted its own evidence.
The scope a query ran inside is recorded as a hash, not a name. A reviewer can still tell two scopes apart and follow rows across a session. The log itself stays clear of becoming a directory of who has access to what.
Money figures in an AI console are computed from token counts and a price table held in the software. That makes them useful for spotting a change in behaviour, and wrong for anything else.
The value of the estimate is comparative. One person’s usage against another’s, this month against last, one tool against the rest. Those readings stay sound even when the absolute figure has drifted from the price list.
The last one sounds like the smallest, and it decides whether anybody believes the other three. A dashboard willing to show zero when there is nothing to show is one you can trust on the day it shows something.
An assertion is a sentence. Evidence is something a second person can check without asking you. What an audit trail has to contain, what a hash actually proves, and what to require of any tool that claims to produce one.
The approve to run rule, in plain words. The assistant reads, explains and proposes, and a person owns the write path. What that costs, what it buys, and three questions to ask of any AI feature.
Two Tableau credentials doing two different jobs. One carries a person’s permissions. The other extends a platform capability. Picking the right one keeps a permission model you spent years building in charge of who sees what.