Isn’t 40 questions too many?

You don’t need all of them in one meeting. Use sections 1–3 in the first call to judge competence, and 4–7 when you’re shortlisting. Most good vendors will be glad you asked.

What if a vendor can’t answer the quality questions?

That’s the most important signal on the list. Without a way to measure accuracy, nobody — including the vendor — knows whether the system works.

Should I ask for references?

Yes, but ask them specific questions too: did the project reach production, did the estimate hold, and could they have switched vendors if needed?

Guide · Buyer’s checklist

40 questions to ask an AI company before you sign.

Choosing an AI vendor on the strength of a demo and a day rate is how expensive surprises start. These questions take an hour in total across two calls. The answers will tell you more than any portfolio — and they work for any vendor, including us.

By Nythrex EngineeringUpdated 24 September 2026 4 min read

Your AI vendor

Approach
Data
Quality
Security
Cost
Ownership
Team

1. Approach: do they solve problems or sell technology?

0/5

2. Data: do they take your data seriously?

0/6

3. Quality: can they prove it works?

0/7

4. Security & compliance

0/6

5. Cost: what am I really paying for?

0/6

6. Ownership: could we leave you?

0/5

7. Team & delivery

0/5

How to read the answers

Warning signs

“Our AI handles that automatically”
Accuracy promises before seeing your data
No mention of evaluation or test sets
Proprietary platform you can’t take with you
Every question answered with “no problem”

Good signs

They ask about your data, users and success criteria
They name risks and how they’d test them early
They talk in numbers: test sets, latency, cost per request
Everything runs in your accounts; you own it all
They tell you where AI isn’t the right tool

Frequently asked questions

Guide9 tricks that make AI demos lieCherry-picked questions, clean sample data, hidden humans, unlimited budgets: how AI demos mislead buyers — and the questions that expose them in five minutes.Guide13 red flags your vendor is hiding problemsVague reports, shifting dates, no access to code: 13 signs your outsourced project is in trouble — and what to ask or do about each this week.GuideWho really owns your code?Paying for software doesn’t make it yours. The clauses that decide code ownership: IP assignment, background IP, open source, AI assets, exit terms.Free toolProject brief generatorFill in nine fields and get a structured software or AI project brief you can send to any vendor — so the estimates you get back are actually comparable.GuideWhat AI development really costsWhat drives AI project cost — the model is the cheap part. Effort by phase, hidden costs vendors skip, running costs and how to budget safely.ComparisonFreelancers vs agency vs in-houseHourly rates hide most of the cost. Freelancers, a development company and an in-house team compared on total cost, speed, risk and control.

Want a second opinion on your project?

Tell us what you’re building and where you’re stuck. We’ll reply within one business day with the most practical next step — even if that step isn’t us.

Start a project