# AI Pilot vs Proof of Concept vs MVP: Which One Do You Need?

URL: https://vrisic.com/blog/ai-pilot-vs-proof-of-concept-vs-mvp/
Last updated: 2026-11-02

**Short answer:** A proof of concept asks “can this be done at all?”, a pilot asks “does it work on our real data and with our real people?”, and an MVP asks “is this first version good enough to run part of the business on?”. Most business AI ideas are technically feasible, so the useful first step is usually a pilot of 2 to 4 weeks on at least 100 real cases, with a pass mark written down before you start.

In this guide

1. What each term means
2. The three side by side
3. What the failure statistics say
4. When a proof of concept is enough
5. What makes a pilot a real pilot
6. How many test cases you need
7. When to build an MVP
8. Illustrative example with numbers
9. A decision rule you can apply today
10. Mistakes we see
11. Write the exit criteria first
12. How Vrisic can help
13. Sources

Proposals for AI work use “proof of concept”, “pilot”, “prototype” and “MVP” as if they were interchangeable. They are not, and the difference decides what you pay, what you learn and whether the thing you receive can be reused. A vendor selling you a “pilot” that is really a demo on ten hand-picked documents has sold you a proof of concept at a pilot price. A vendor selling an “MVP” with no test set has sold you a guess.

This guide gives plain definitions, the questions each stage answers, realistic durations and costs, the sample sizes you need before a number means anything, a worked example, and a decision rule. We sell an [AI Pilot](https://vrisic.com/ai-pilot/) , so we have a stake in the answer; where a proof of concept or a straight build is the better choice we say so.

## What do proof of concept, pilot and MVP each mean?

The cleanest way to separate them is by the question each one answers. The innovation charity Nesta, which has written about the confusion, frames it the same way: a proof of concept shows whether something *can* be developed, a pilot is a live rollout to a small group of real users to understand how it works in reality, and an MVP tests only the essential core of an idea with real users to find out whether there is an actual need for it ([Nesta](https://www.nesta.org.uk/blog/proof-of-concept-prototype-pilot-mvp-whats-in-a-name/) ).

- **Proof of concept (PoC):** a small, throwaway experiment that tests one technical assumption. Example: “can a model read our scanned delivery notes at all?” Sample data is fine. Nobody outside the project team uses it.
- **Pilot:** the real workflow, run on real data, for a limited time and a limited group, with results measured against a pass mark. Example: “over three weeks, does the system classify our own inbound emails correctly at least 90% of the time, and do the three people who triage them find it faster?”
- **MVP (minimum viable product):** the first version that is actually deployed, integrated with your systems, secured and supported, even though it does only one thing. It is built to be used and then extended, not thrown away.

Two terms you will also meet. A **prototype** shows how something will look and behave, and is mostly about interface and flow rather than accuracy. A **beta** is an MVP released to a wider but still limited audience. Neither changes the core logic: you move from “can it work” to “does it work here” to “can we run on it”.

Three stages, three different questions. Vrisic analysis; definitions after Nesta

## How do the three compare side by side?

The table below uses typical figures for a single task AI project of the kind we build, such as email triage, document extraction, lead qualification or an internal knowledge assistant. Treat durations and costs as ranges; they move with integration depth and data quality.

|  | Proof of concept | Pilot | MVP |
| --- | --- | --- | --- |
| Question it answers | Can it work at all? | Does it work for us, on our data and people? | Can we run part of the business on it? |
| Data used | Sample, public or synthetic | A representative slice of your real data | Live production data |
| Users | The project team | A few named staff, in their normal work | Everyone in the target process |
| Integrations | None, or mocked | Read access, or a safe sandbox | Full, with write access and approvals |
| Typical duration | A few days to 2 weeks | 2 to 4 weeks | 3 to 10 weeks |
| Typical cost at Vrisic | Usually folded into scoping | $1,500 for the AI Pilot | $8,000 to $25,000 single task; $25,000 to $60,000 integrated |
| Output you keep | A finding and a short note | Accuracy numbers, a test set, working code | Production software, runbook, monitoring |
| Is the code reusable? | Rarely | Often, as the core of the build | Yes, it is the product |
| Main risk it removes | Technical impossibility | Poor fit with your data and process | Operational and adoption risk |

Many AI proofs of concept never ship. Sources: Gartner; CIO (IDC); CIO Dive (S&P Global)

## What do the failure statistics say about AI projects?

Four widely quoted numbers explain why this choice matters. Read them with their caveats, because each measures something slightly different.

| Finding | Source and date | What it actually measures |
| --- | --- | --- |
| At least 30% of generative AI projects abandoned after proof of concept by end of 2025 | Gartner prediction, July 2024 | A forecast, not a count. Gartner names poor data quality, inadequate risk controls, escalating costs and unclear business value as the causes. |
| 4 of every 33 AI proofs of concept reached production (about 12%) | IDC research with Lenovo, reported March 2025 | Observed PoCs at the companies studied. The article does not give a sample size. |
| 42% of companies abandoned most of their AI initiatives in 2025, up from 17% in 2024 | S&P Global Market Intelligence survey of 1,000+ respondents in North America and Europe | Self reported. The same survey found companies scrapped an average of 46% of proofs of concept before production. |
| Over 40% of agentic AI projects cancelled by end of 2027 | Gartner prediction, June 2025 | A forecast. Reasons: escalating costs, unclear business value, inadequate risk controls. |

You may also have seen “95% of AI pilots fail”, from an MIT report. That figure covers 300 projects that were not randomly sampled and measures “no measurable P&L impact”, which is not the same as a failed pilot; many of the projects had no documented baseline to measure against ([Agent Mode AI analysis](https://agentmodeai.com/the-mit-genai-pilot-failure-claim/) ). We do not use it as evidence in either direction.

The pattern across the credible numbers is consistent. Projects rarely die because the model cannot do the task. They die because data is not ready, value was never defined, risk controls were missing, or costs rose once the real work began. A proof of concept can only address the first of these, and only partially. A well designed pilot addresses all four, which is why we recommend it as the default first step. IDC’s Ashish Nadkarni makes a related point in the CIO article: many initiatives start under board pressure and the proofs of concept are underfunded or unfunded ([CIO](https://www.cio.com/article/3850763/88-of-ai-pilots-fail-to-reach-production-but-thats-not-all-on-it.html) ).

## When is a proof of concept enough?

A proof of concept is the right tool when there is a genuine technical unknown that is cheap to test and expensive to discover late. It is the wrong tool when the unknown is business fit, because it cannot answer that.

### Good reasons to run a PoC

- **Unusual inputs.** Handwritten forms, engineering drawings, low quality call audio in a mixed language, or very long contracts where it is unclear that any model will cope.
- **A hard constraint.** The model must run inside your network with no external API, and you want to know whether an open weights model is good enough before committing to hardware. See our guide to [private and self hosted LLMs](https://vrisic.com/blog/private-llm-self-hosted-ai/) .
- **A new capability.** You want to know whether an agent can complete a multi step task using a specific tool or API before designing a workflow around it.
- **A fast kill test.** You suspect the idea will fail and want to find out in three days, not three months.

### What a PoC must not claim

A PoC run on 10 to 20 chosen examples can show that something is possible. It cannot show an accuracy rate, a time saving or a return, because the sample is too small and usually flattering. If a vendor reports “95% accuracy” from a PoC, ask how many cases, who chose them and how many were not seen before the demo. The table in the next sections shows why a small sample cannot support a claim like that.

Many business ideas do not need a PoC at all. Reading invoices, classifying emails, answering questions over a document set and drafting replies from a CRM record are well understood. Spending two weeks proving them feasible is spending money to learn what is already known; go straight to a pilot.

## What makes a pilot a real pilot?

A pilot is real when it can fail. That means five things are fixed in writing before any build starts. If any is missing, you are running an extended demo.

1. **A baseline.** What does the process cost today in minutes per item, error rate and cycle time? Without it you cannot show an improvement. Measure two to four weeks of real work, or sample 50 items and time them.
2. **A representative test set.** Real items drawn from the last few months, including the awkward ones, with the correct answer written by the person who owns the process. Do not let the vendor choose them. Keep part of the set hidden from the build team until the final run.
3. **A pass mark.** For example, “at least 90% of fields correct, no more than 1 in 100 wrong values passed as confident, and median handling time down by 40%”. Choose it from the cost of an error, not from what the demo achieved.
4. **Real users in real conditions.** Two to five people who do the job today use the output for part of their working day. Their corrections are data, and their objections are findings.
5. **Kill criteria and a decision date.** The conditions under which you stop, and the date on which someone with budget authority decides. A pilot with no decision date becomes a permanent experiment.

The output of a good pilot is not software. It is a one page result: accuracy per field or per category, the error types, the cost per item, the time saved in the users’ own words, and a go, change or stop recommendation with a fixed price for the next stage. The code is a by-product, though a useful one: in our experience it is usually the starting point of the build rather than a throwaway.

### Silent errors matter more than the headline accuracy

Report two error types separately. A *flagged* error is one the system knew it was unsure about and sent to a person. A *silent* error is a wrong answer delivered with confidence and acted on. A system with 92% accuracy where every wrong answer is flagged is safer than one with 96% accuracy where the wrong answers are invisible. Your pass mark should cap silent errors explicitly.

Ten test cases cannot prove 90% accuracy. Vrisic calculation, Wilson score interval, 95% confidence

## How many test cases do you need before a number means anything?

Fewer than most people think are needed to rule an idea out, and more than most pilots use to prove one. The reason is ordinary statistics. If a system gets 90% of your test cases right, the true rate on all your future cases could be noticeably higher or lower, and the uncertainty shrinks slowly as the sample grows.

The table shows the 95% confidence interval (Wilson score method, our own calculation) around an observed 90% accuracy at different test set sizes.

| Test cases | Observed correct | True accuracy is probably between | Width of the range |
| --- | --- | --- | --- |
| 10 | 9 | 59.6% and 98.2% | 38.6 points |
| 20 | 18 | 69.9% and 97.2% | 27.3 points |
| 50 | 45 | 78.6% and 95.7% | 17.1 points |
| 100 | 90 | 82.6% and 94.5% | 11.9 points |
| 200 | 180 | 85.1% and 93.4% | 8.3 points |
| 300 | 270 | 86.1% and 92.9% | 6.8 points |

Three practical conclusions follow.

- **A 10 or 20 item demo proves almost nothing about accuracy.** “9 out of 10 right” is compatible with a true rate of 60%.
- **100 cases is the working minimum for a go decision** on a single category task. At 100 cases a result of 92 correct puts the lower bound at about 85%.
- **Per category numbers need their own cases.** If you have five document types and want a reliable figure for each, you need roughly 50 to 100 of each, or you must accept wide ranges on the rare ones and route them to people by default.

If your volume is low, say 30 items a month, you cannot collect 100 fresh cases in a pilot window. Use historical items for the test set, and run the live part of the pilot for user experience and time saved rather than for accuracy.

## When should you build an MVP?

Build an MVP when the pilot cleared its pass mark, a named owner will use the result, and you can describe the single process it will run. The word “minimum” applies to scope, not to engineering quality: an MVP touches real systems and real money, so authentication, logging, error handling and a human approval step for risky actions are in scope from day one.

### What an MVP includes that a pilot did not

- **Integration with write access.** Creating the CRM record, posting the bill, sending the reply, with an approval step where the cost of a mistake is high. See our notes on [integrating AI with your CRM](https://vrisic.com/blog/ai-crm-integration-guide/) .
- **Monitoring.** Per-item logs, a sampled review queue, drift alerts when accuracy slips, and cost tracking for model usage.
- **Fallbacks.** What happens when the model API is down, when the input is unreadable, or when confidence is low.
- **Security and access.** Role based access, secrets management, data retention rules and a record of who approved what.
- **Handover.** Documentation, a runbook and a test suite so that your team, or any other vendor, can maintain it.

At Vrisic a single task agent typically costs $8,000 to $25,000 and takes 3 to 6 weeks; an agent integrated into several systems typically costs $25,000 to $60,000 and takes 6 to 10 weeks. You receive one fixed price in writing after a free scoping call, and the price assumes a pilot-quality understanding of the data. For wider cost drivers see [what AI agents cost](https://vrisic.com/blog/ai-agent-development-cost/) and the broader [AI development cost guide](https://vrisic.com/blog/ai-development-cost/) .

### When to skip the pilot and go straight to an MVP

Skipping the pilot is reasonable when all of these hold: the task is a standard one with well documented model behaviour, your data is clean and already labelled, the cost of a wrong answer is low or fully caught by review, and the build price is small relative to the pilot. In that case a first milestone of the build can play the role of a pilot, as long as a go or stop point with a pass mark is written into the plan.

## Illustrative example: quote request intake

This is an illustrative example with assumed numbers, not a client project. A distributor receives 600 quote requests a month by email. Each is read, matched to products and typed into the CRM by a sales coordinator in about 12 minutes. Fully loaded cost is $40 an hour.

**Today:** 600 requests x 12 minutes = 120 hours a month = $4,800 a month.

**The idea:** an agent reads each email, extracts customer, products and quantities, drafts the CRM record and flags anything uncertain.

### Step 1: skip the PoC

Extraction from emails is a standard task, so there is no technical unknown worth a separate experiment.

### Step 2: run the pilot

The team pulls 150 past requests, including attachments in PDF and spreadsheets and a dozen messy ones, and writes the correct CRM entry for each. Pass mark: at least 90% of fields correct overall, no more than 1 silent error per 100 requests, and at least 50% of requests handled with only a quick check. Result in this example: 91% of fields correct, 2 silent errors in 150 requests, 58% of requests needing only a check, and a coordinator reports 2 minutes per checked request instead of 12.

This clears two of the three marks and misses the third. Silent errors came from two requests where the customer gave a part number that appears in two product families. The pilot recommendation is “change”: add a rule that flags any ambiguous part number, then re-run the 150 cases. After the fix the silent errors drop to 0 in the test set. This is exactly the discovery a demo would have hidden.

### Step 3: build the MVP

Assume a fixed price of $18,000 for a single task agent writing into the CRM with an approval step, and $400 a month for hosting and model usage.

| Line | Figure | Working |
| --- | --- | --- |
| Hours after launch | About 62 a month | 600 x (58% x 2 min + 42% x 12 min) = 600 x 6.2 min = 3,720 minutes |
| Hours saved | About 58 a month | 120 less 62 |
| Value of hours saved | About $2,300 a month | 58 hours x $40 = $2,320 |
| Running cost | $400 a month | Hosting and model usage |
| Net monthly benefit | About $1,900 | $2,320 less $400 = $1,920 |
| Payback on $18,000 total | About 9 to 10 months | $18,000 / $1,920 = 9.4 months |

That is a modest return, and the honest conclusion is that the case rests on the coordinators’ time being redeployed to selling, not on headcount cuts. If the pilot had shown only 30% of requests handled with a quick check, the monthly benefit would have been under $1,000 and the business case would not have held at $18,000. That is the point of a pilot: finding that out for $1,500 rather than $18,000.

### The expected cost of skipping the pilot

Suppose, as an assumption for this example, that there is a 30% chance the build would miss its pass mark without a prior test (Gartner’s forecast of 30% abandoned after PoC is a rough external reference). Expected waste without a pilot = 30% x $18,000 = $5,400. With a pilot, the downside is capped at the pilot fee of $1,500. Even if the true chance of failure were only 10%, expected waste is $1,800, still above the pilot cost. The pilot only loses on value when the build is cheap, the risk is very low, or you can fully credit the pilot against the build, which is our model.

**Get your own pass mark and fixed price in 10 business days** Our AI Pilot runs on your real data against a pass mark we agree first, with the $1,500 fee credited if you build within 60 days.

[See how the AI Pilot works](https://vrisic.com/ai-pilot/)

## Which should you choose? A decision rule

Answer these questions in order and stop at the first “yes”.

| If this is true | Do this | Why |
| --- | --- | --- |
| You do not know whether any model can read, hear or decide on your inputs | Proof of concept, 2 to 5 days | A cheap kill test for a technical unknown |
| The task is standard but you do not know your accuracy, cost per item or user acceptance | Pilot, 2 to 4 weeks | Measures fit on real data against a pass mark |
| The pilot cleared its pass mark and an owner is ready | MVP, 3 to 10 weeks | Turns evidence into production software |
| The task is standard, data is clean, errors are cheap and caught by review, and the build is small | Straight to MVP with a pilot style first milestone | A separate pilot would cost more than the risk it removes |
| A packaged tool already does 80% of the job | Trial the tool for a month before commissioning anything | Cheapest evidence; see [build vs buy](https://vrisic.com/blog/build-vs-buy-ai-software/) |
| You cannot name an owner, a metric or a baseline | None yet. Fix that first | No stage can succeed without a definition of success |

The last row is the one most often ignored. If nobody owns the outcome, a pilot produces a report, an MVP produces software, and neither changes how work is done. Our AI readiness checklist covers the prerequisites.

## What mistakes do teams make with pilots, PoCs and MVPs?

- **Choosing the demo set.** Examples picked by the builder are easier than your real workload. Always test on a set you or the process owner assembled.
- **Reporting an average.** “92% accurate” hides that totals are right 99% of the time and dates 70%. Report per field and per category.
- **Measuring the model, not the process.** A model that is 95% accurate but needs a slow approval chain saves no time. Measure end to end minutes per item.
- **No baseline.** Without today’s minutes and error rate you cannot prove a gain, which is also the main weakness of the “95% of pilots fail” headline.
- **Treating the pilot as a free trial of vendor effort.** A pilot with no fee tends to get the junior team and the shortest attention. A small fee, credited against the build, aligns incentives.
- **Letting the pilot run forever.** Without a decision date, enthusiasm fades and the pilot lingers as a permanent experiment. Put the decision meeting in the calendar on day one.
- **Throwing away the pilot code, or the opposite.** Pilot code is normally a good base, but it has no monitoring, error handling or access control. Plan time to harden it.
- **Scaling cost surprise.** Model usage at pilot volume looks trivial. Estimate cost per item at ten times the volume before you commit, especially for agents that make several model calls per item.
- **Scope creep at MVP.** Adding a second workflow “while we are at it” turns a 4 week build into a 12 week one. Ship the one process, then extend.

## What should you write down before the first day?

One page, agreed by the process owner and the budget holder. If it takes longer than an hour to write, the project is not ready.

1. **The process in one sentence** and who owns it.
2. **The baseline:** items per month, minutes per item, error rate, cost per item.
3. **The test set:** where it comes from, how many items (at least 100), who writes the correct answers, and which part is held back.
4. **The pass mark:** accuracy, silent error cap, time saving, and cost per item including model usage.
5. **Kill criteria:** the result at which you stop, for example “below 80% accuracy after two improvement rounds”.
6. **Data and security limits:** what data may be used, where it is stored, who can see it, and what is deleted at the end.
7. **The next step and its price:** what you will be offered if the pilot passes, and the date of the decision meeting.

If a vendor will not agree a pass mark or a fixed next-stage price before you start, treat that as information about how the build will go. For more on judging suppliers, see [how to choose an AI development company](https://vrisic.com/blog/how-to-choose-an-ai-development-company/) and the stage by stage view in our [AI development process and timeline](https://vrisic.com/blog/ai-development-process-timeline-stages/) guide.

## How Vrisic can help

Our [AI Pilot](https://vrisic.com/ai-pilot/) is a pilot in the sense used above: $1,500 (£1,200, AED 5,500), 10 business days, run on your own data against a pass mark we agree first. The fee is credited in full against a build if you go ahead within 60 days. If the numbers do not support a build, we say so and you keep the test set and the findings.

If you go ahead, builds are fixed price after a free scoping call. They run in your cloud account with your own model keys, you own the code, prompts and test data, and there are no per seat fees. Read [how we work](https://vrisic.com/how-we-work/) , or [book a scoping call](https://vrisic.com/contact/) to talk through which stage fits your project.

## Sources

1. [Proof of concept, prototype, pilot, MVP: what’s in a name?](https://www.nesta.org.uk/blog/proof-of-concept-prototype-pilot-mvp-whats-in-a-name/) (Nesta)
2. [Gartner Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept By End of 2025](https://www.gartner.com/en/newsroom/press-releases/2024-07-29-gartner-predicts-30-percent-of-generative-ai-projects-will-be-abandoned-after-proof-of-concept-by-end-of-2025) (Gartner, 29 July 2024)
3. [88% of AI pilots fail to reach production, but that’s not all on IT](https://www.cio.com/article/3850763/88-of-ai-pilots-fail-to-reach-production-but-thats-not-all-on-it.html) (CIO, 25 March 2025, reporting IDC research with Lenovo)
4. [AI project failure rates are on the rise: report](https://www.ciodive.com/news/AI-project-fail-data-SPGlobal/742590/) (CIO Dive, reporting S&P Global Market Intelligence)
5. [Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027](https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027) (Gartner, 25 June 2025)
6. [MIT 95% AI pilot failure: what the GenAI Divide measured](https://agentmodeai.com/the-mit-genai-pilot-failure-claim/) (Agent Mode AI)

Confidence intervals, the worked example and the expected cost figures are Vrisic’s own calculations from stated assumptions, not published benchmarks. Statistics from third parties were checked in October 2026.

## Frequently asked questions

### What is the difference between an AI pilot and a proof of concept?

A proof of concept tests one technical assumption, such as whether a model can read your scanned forms, usually on a handful of sample documents and for a few days. A pilot runs the real workflow on a representative slice of your own data, with a few real users, for two to four weeks, and judges it against a pass mark agreed beforehand. A PoC shows something is possible; a pilot shows it is worth building.

### How many test cases do I need for an AI pilot?

At least 100 real cases for a single category task. With 10 cases, a result of 9 correct is statistically consistent with a true accuracy anywhere from about 60% to 98%. With 100 cases and 90 correct, the 95% range narrows to roughly 83% to 94%, and at 200 cases to 85% to 93%. If you need figures for several document types, plan for 50 to 100 of each.

### How much does an AI pilot cost compared with an MVP?

Vrisic’s AI Pilot is $1,500 (£1,200, AED 5,500) for 10 business days, credited in full against a build started within 60 days. A typical MVP, meaning a first production version, costs $8,000 to $25,000 for a single task agent and $25,000 to $60,000 for an agent integrated with several systems. A proof of concept is usually a few days of effort and is often folded into scoping.

### Can I skip the proof of concept and go straight to a pilot?

Yes, and for most business tasks you should. Reading invoices, classifying emails, extracting data from documents, answering questions over a knowledge base and drafting replies are well understood, so there is no feasibility question to prove. Run a proof of concept only when inputs are unusual, such as handwriting or poor audio, or when a hard constraint like no external API makes model quality uncertain.

### Is an MVP the same as a pilot?

No. A pilot is a time limited test whose output is evidence and a recommendation. An MVP is the first production version: integrated with your systems, secured, monitored and supported, with a human approval step for risky actions. Pilot code is often reused as the core of the MVP, but it needs hardening: logging, error handling, access control, fallbacks and documentation before real users depend on it.

### What should I do if the pilot misses its pass mark?

Read the failure types before deciding. If errors cluster around something fixable, such as ambiguous part numbers or one poor-quality input source, change the rules or inputs and re-run the same hidden test set. If accuracy stays under your kill threshold after two improvement rounds, or errors are silent and spread across categories, stop. You keep the test set, the baseline and the findings, which are reusable for any later attempt.

### Should an AI pilot be free?

We think a small fee is healthier. Free pilots tend to get less senior attention and shorter timelines, and customers have less reason to assemble a proper test set and baseline. A modest fee that is credited against the build keeps both sides serious and costs you nothing extra if you proceed. Whatever you choose, agree the pass mark, the data you will provide and the decision date in writing.

---
Vrisic · https://vrisic.com/ · Contact: info@vrisic.com · Book a call: https://vrisic.com/contact/