Building an AI sales assistant for a GTM team in Manufacturing.

The context

The client designs and makes specialised electronic equipment. It has about 100 people, belongs to a larger international group and sells in dozens of countries. Research customers buy on relationships and expertise. A wider, fragmented market of engineering firms and contractors buys through distributors. A sales team of five at head office is now joined by new reps abroad and more distributors.

The problem

At intake, my sponsor described the problem as knowledge transfer, with four symptoms:

• Technical questions end up with R&D, because there’s no product management team to answer them.
• Distributors ask the same things every week: which cable fits, which product version, how to configure a quote.
• New reps abroad and new distributors need the same answers, in three languages, without a colleague at the next desk.
• Much of the know-how sits with a few long-serving people and isn’t written down.

Reps need exact specifications for emails and tenders. Some answers sit deep in long PDFs, and some aren’t written down anywhere. A guessed figure or certification in a tender becomes a promise to the customer, so the assistant had to cite its sources and say when the documents don’t answer.

The app and its code are private. The client and its products are unnamed here.

The solution

A rep asks in their own language, gets an answer from the product documents with each fact linked to its source, and can turn it into a deck, a tender table or an email. Five choices shape it:

• Every figure is cited, and a tender line the documents don’t cover reads “unknown”, so nobody commits to a figure nobody found.
• A second model checks each claim and code sets the badge, so a rep knows how far to trust an answer before reusing it.
• Each gap goes to the product team with the document to fix, and an approved note becomes citable, so know-how gets written down.
• Drafts copy figures exactly and turn a missing price or lead time into a question to confirm. Nothing reaches a customer until a rep sends it.
• Every model call goes through one gateway, so changing a model is one line of configuration, and production would search the company’s SharePoint with its permissions.

I ran the engagement from the first call to the pilot: discovery and the use-case roadmap with the sponsor, the architecture, the security review, the evaluations and the rollout plan. Coding agents wrote much of the code from plans I approved, and I reviewed every change before it merged.

How it works

The pilot runs on Next.js. Claude models route, write and check answers, and an OpenAI model embeds the passages, all through Vercel AI Gateway. A question goes through five steps:

1. 01
   Routing. A small model picks a light, standard or deep answer model. Code first computes cheap signals: the language, product names, comparison words and price words. Any failure falls back to the standard model. A faster router has its own note.
2. 02
   Search. The answer model can search up to seven times, in one index file built offline from about 740 documents, 25 of them support pages behind the client’s password. Each search ranks passages by keywords and by meaning, merges the two lists with reciprocal rank fusion, and returns eight passages, no more than two from one document.
3. 03
   Answer. The model must cite a passage for every fact and say when the documents don’t cover the question. Prices and lead times go to sales administration. The rep can turn the answer into a deck, a tender table or an email.
4. 04
   Check. After the answer appears, a separate model splits it into claims and labels each one against the passages. Code turns the labels into the badge the rep sees.
5. 05
   Record. Each model call writes one entry to an audit log in the EU, after the response, so a logging failure can’t affect the answer. Weak answers go to the gap queue.

Trust and access fail closed: if sign-in or the check fails, the app refuses or says so. Metering and the audit log fail open, so an outage there never blocks an answer.

How far to trust an answer

Code sets the badge from the judge’s labels, because a model’s own confidence runs high. “Verified” needs 90% of claims supported and none unsupported, every citation must match the index word for word, and a failed check never shows a positive result. The badge measures support in the documents, not truth. Try the rule:

Feedback showed the badge must also say what’s missing. The sponsor saw “medium confidence” over 10 supported claims, so coverage got its own line. A rep then got a green badge over an answer that couldn’t help, ending in advice to ask the technical team. Now a gap leads with an amber “Incomplete” banner naming what’s missing, says it went to the documentation team only when a gap was really logged, and sends the question to the technical team in one tap. That change is built and in review.

Evals

Each step a model drives has its own eval and a fixed idea of what right looks like. Code grades it where the answer is known. Where it isn’t, a model grades it, and that model needs checking against people.

• Routing: 47 labelled questions in three languages, run three times and graded by exact match. A Claude model and an OpenAI model each routed correctly every question they returned a label for, so I kept the faster one. One case passed for the wrong reason: the router failed and the fallback happened to pick the expected model. The eval now fails on any invalid router output.
• Search: the model rewrites a question before it searches, so tests on users’ own words missed real failures. Replaying the 51 searches it actually made found four that missed the right passage, and a ranking change fixed three. Searching the English-only documents too raised the right page found from 10 to 15 of 20 test questions on the light model.
• Claims: the judge labels every claim of every answer in production, and code turns the labels into the badge.
• Code: unit tests cover the badge rules, citation checks, conversation trimming and sign-in gates on every pull request.

Not done yet: the judge, which grades every answer, hasn’t been checked against human labels. A 21-case prompt-injection set in three languages is built but hasn’t run, and there’s no labelled set for search recall.

Security and data

Before the pilot widened, I wrote a threat model: 14 threats across eight trust boundaries, each with a mitigation and the risk left over. Two days and 15 merged changes later, I audited the app against it again. Self sign-up is off, and super admins are named in configuration, so nobody can grant that role inside the app. A database trigger refuses any edit or deletion in the audit log; production will also need a separate write-only role for it.

What the assistant copies, who can see each copy, how retrieved text is kept from steering the model, and the per-person spending cap have their own note. Customers’ names, email addresses and phone numbers are masked before they leave the browser, and every model call asks for zero data retention. Pinning every model call to the EU and a retention rule for the log are built and tested, not yet in production. Some findings are still open, and their details stay off this page.

The initial pilot

• Most questions came in French (13) or English (9), and none in Spanish. People made 3 decks, 1 tender table and no emails.
• 18 answers showed a missing, incomplete or conflicting document. The product team hadn’t reviewed any of them yet.
• The source check marked 14 answers fully supported. Nobody has measured yet how often it agrees with a human reviewer.
• Routing took about two seconds of the 5.3-second wait.

Routing didn’t save money yet: sending 26% of answers to the deep model cost 25% more than an estimate of using the standard model for every question. Trimming old search results and reusing repeated text cut cost by 39–47% in a scripted four-question test, and by less on the shorter real conversations.

What a full rollout would add

The pilot answers from public documents for a few invited users. Moving it onto the company’s own documents adds five pieces of work:

• Sign-in through the company’s Microsoft directory, so accounts are added and removed there. It’s built and limited to an allowlist of directories, but hasn’t met a real one yet.
• Search over the company’s SharePoint that keeps its permissions: passages a person can’t open are dropped before ranking, so the model never sees them. I’ve prototyped this separately; it isn’t in this app.
• A write-only role for the log, the retention periods, and consultation with employee representatives. Test and production data are already separate, and EU-only models and the retention rule are built.
• A daily cost cap per person instead of a request-rate limit, because one tender can mean 150 requests in a few minutes. It went live in production on 1 November 2026.
• Adoption: at intake the sponsor expected changing habits to be harder than the technology, and reps’ experience with digital tools varies. Seven people used the initial pilot, and nobody owns the gap queue yet. A wider rollout needs a named owner inside the company, a few volunteer reps using it on live work, and a weekly look at who uses it.

Lessons

The sponsor asked for a central knowledge base before any use case. Building the assistant first turned out to be a way to get one: the 18 flagged answers are a content backlog for the product team as much as help for reps. Nobody had reviewed them yet, and only seven people had used it, so the first risks to fix are an owner for the gaps and regular users.

The pattern is common in manufacturing. Product knowledge sits in PDFs in several languages, and the costly mistakes are specifications and certifications.

The next test needs human checks of answers and source links, the prompt-injection set run through the whole app, and enough regular users to judge adoption. Before keeping the router as it is, I’d also measure whether the expensive model gives better answers.

The figures on this page come from a small initial pilot in October 2026. Later code changes are not counted as pilot results. Client names, products, documents and conversations are withheld.