Why we stopped building one-off dashboards

Ask a chat tool for a sales dashboard and you will have one in under a minute, with tidy charts, a few headline numbers and perhaps a filter or two. For a quick look at a spreadsheet that is useful, and I do it myself.

The trouble starts when an organisation begins to rely on those dashboards, which is why we have stopped building one-off dashboards for clients. Instead, we build what sits underneath, so that AI can make good ones over and over again.

What goes wrong with a generated dashboard

Every one starts from nothing. Each time someone asks, the tool reads the data afresh, guesses what the columns mean and decides how to work out each figure, and nothing carries over from the last time.

The numbers drift. If you ask for “revenue this quarter” on Monday and again on Thursday, you may well get two different figures, because one included refunds and the other did not, or one went by order date and the other by invoice date. Both will look equally sure of themselves.

Definitions do not agree. Finance means one thing by “active customer” and sales means another, and the AI picks whichever reading seems most likely. When two departments bring different numbers to the same meeting, the conversation turns to whose chart is right instead of what to do about it.

You cannot drill down. A generated chart is usually a dead end: you cannot click a bar to see the records behind it, and you cannot trace a figure back to where it came from.

I do not think any of this is the AI being bad at its job. It is being asked to make the same decisions from scratch every time, with no memory of what the organisation has already agreed.

What we build instead

We build five things, and then the client’s own AI tools do the drawing.

1. A clean data model with provenance. There is one place where the data lives, structured sensibly, and every figure can be traced to its source, so that when someone asks where a number came from there is an answer.

2. Agreed metric definitions. “Revenue”, “active customer” and “churn” are each defined once, in writing, with the calculation attached. The AI uses the definition it is given instead of making one up.

3. A reusable component library. We provide charts, tables and filters that already know how to drill down from a summary to the detail underneath, so a new dashboard is assembled from parts known to work rather than drawn freehand.

4. Instructions for your AI tools. This is clear guidance, kept up to date, which tells whichever assistant your team uses which data, definitions and components to work with, and what to do when a question falls outside them.

5. A learning loop. This is the part a chat window cannot do. When the AI produces a result, the people who know the subject (analysts, finance leads and operations managers, for example) rate how confident they are that it is right, and say why when it is not. Those ratings feed back into the definitions, the instructions and the examples the AI works from.

Why the learning loop matters most

A chat window starts from scratch every time, whereas the system we build improves the more it is used. Each rating from an expert makes the next answer a little more likely to be right, and the corrections stay with the organisation instead of vanishing when someone closes a chat.

It also keeps people’s judgement at the centre of the work. The AI does the repetitive job of putting queries and charts together, and the people who understand the business decide what is correct. Over time, the questions that need an expert’s attention become the difficult and interesting ones, and the system handles the ones it should already know.

The dashboards we still build

None of this means dashboards have had their day. Where we build software that gathers its own data, such as the aviation assessment platform, we still design the dashboard ourselves, because the questions are largely known in advance and asked again and again. That dashboard can be trusted for the same reasons set out above: the data is structured as it is collected, every figure is defined once, and any chart can be opened to show the records behind it. I have written about it in a separate article.

What we have stopped doing is building one-off dashboards on top of data that was never prepared for them, where the questions change from week to week and no hand-built dashboard could keep up. There, AI should do the drawing, as long as it has the foundations to draw from.

What this means day to day

Teams still ask questions in plain language and still get a dashboard back in seconds. The difference is that Monday’s figure matches Thursday’s, finance and sales work from the same definition, any number can be traced back to where it came from, and the whole thing improves with use instead of starting again each time.

We call this an Insight Platform. If your organisation already uses AI for reporting and the numbers do not quite agree, I would be glad to talk it through with you.