Where does the language model run, and how do you check its answers?

A question to a language model is rarely innocent. “Which accounts did this company pay in March?” already tells anyone who reads it which company is being looked at. Send that question to a model hosted somewhere else, and you send it to someone else’s machines, often with the documents that explain it.

Where the model runs

For a government department, a lender or a hospital, where the model runs is not a technical detail. Anna AI is part of the platform itself and runs on machines you control. It reaches no outside service, and what your teams type into it does not leave the building. What you ask can say as much as what you are told. Here, both stay on your side.

An answer is not proof

Where the model runs decides who sees the questions. It does not decide whether the answers are right.

In February 2024, a tribunal in British Columbia ruled on a dispute that began with a chatbot. A traveller asked the assistant on Air Canada’s website about reduced fares for people travelling after a death in the family. It told them they could claim the reduced fare after the trip, within 90 days of buying the ticket. The reply even linked to the airline’s own page on the subject, and that page said the opposite: the reduced fare could not be claimed once the trip was over. Air Canada argued that it could not be held liable for its chatbot. The tribunal called this “a remarkable submission” and wrote that “it should be obvious to Air Canada that it is responsible for all the information on its website”.

Pointing to sources helps, but it is not enough on its own. In 2024, researchers tested commercial legal research tools that look up sources before they answer, following a method they had published in advance. One of the tools was sold as giving “hallucination-free” legal citations, meaning it would never make one up. They found that the tools “each hallucinate between 17% and 33% of the time”.

We draw two lessons from this. The organisation answers for what its model says. And a link to a source is worth nothing until someone reads it.

Every answer, its records

Anna AI draws its answers from your organisation’s own records and case files, and stays within what they say. Each reply names the records it rests on, so you can trace it back to where it came from.

We think the right test of a language model, inside an organisation, is not how well it writes but how quickly a person can check it. When an answer names its records, the question “is the model right?” becomes “do the records say so?”, and anyone on the case can answer that.

Figure 1

Check the answer

Interactive

Asked to the AI model

When was the grant to Harbour Lane Supplies paid, and to which account?

The model’s answer

0 of 4 sentences checked

• The grant was paid on 21 March 2026.

  Payment order PO-2291Paid on 12 March 2026

  Not checked

• It went to the account ending 4471.

  Payment order PO-2291To the account ending 4471

  Not checked

• That account is held by Harbour Lane Supplies Ltd.

  Bank recordAccount holder: Harbour Lane Supplies Ltd

  Not checked

• The company was registered in 2023, at 5 Harbour Road.

  Company registerRegistered in 2023, at 5 Harbour Road

  Not checked

Try itCheck the answer against its records and watch each sentence.

Check against the records

The answer reads well. That proves nothing until each sentence is checked.

The model drafts, a person checks

No language model is right every time, and we do not claim that ours is. What we offer is an answer whose weak points can be found: a sentence that none of the named records supports, or a record that says something else. Before anyone acts, the person who will act opens the records. It is a small step, and it is the one that turns a well-written reply into a fact the organisation can stand behind.

How to check it

1. Ask Anna AI a question whose answer you already know, and open every record the reply names.
2. Ask a question your records cannot answer, and look closely at what the reply rests on.
3. Give a reply to a colleague who did not ask the question, and see whether they can check it from the named records alone.

Next on the record · #5How can you tell whether an audit trail has been altered?

Sources

1. Moffatt v. Air Canada, 2024 BCCRT 149, Civil Resolution Tribunal of British Columbia, decision of 14 February 2024, paragraphs 15 to 17 and 27. Civil Resolution Tribunal
2. Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D. Manning and Daniel E. Ho, “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools”, 2024. arXiv