Lesson 3: The Jagged Frontier — Claims vs Reality
By the end of this lesson, you will be able to:
- Explain what a benchmark, a demonstration, and a capability announcement do and do not tell you about a platform's fit for your project (S25)
- Rank sources of platform information by reliability, from primary documentation to marketing material (S25)
- Explain the jagged frontier and why AI capability does not track task difficulty as humans perceive it (K25)
- Place a validation step where the frontier makes one necessary, and explain why the shifting frontier makes staying current a professional necessity (K25, S25, B6)
- Test a vendor claim against what your Companion actually does, then apply that scepticism to a claim about a candidate platform, verified against primary documentation (S25, K25)
Claims are made to sell
You have committed to a platform on paper. Before you finalise that recommendation, there is a habit to develop that protects every decision you will ever make in this field: reading claims critically, and knowing where a tool's capability actually stops.
The AI and automation market moves fast and markets itself hard. Vendors publish benchmark scores, record polished demonstrations, and announce new capabilities continually. None of this material is neutral. It is produced to sell, and a practitioner who takes it at face value will make decisions on evidence that was never meant to be evidence. The skill this lesson builds is not cynicism. It is the ability to extract the useful signal from vendor material while recognising what it conveniently leaves out — and, underneath that, to understand why even an honest claim can mislead you about your own project.
Aim for calm scepticism here, not hostility. Vendors are not lying to you; they are presenting their best case, which is what marketing is for. Your job is to know what that best case leaves out and go looking for it — not to dismiss vendors as untrustworthy. Swinging into blanket distrust leaves you as poorly calibrated as believing everything.
What benchmarks measure, and what they do not
A benchmark is a standardised test that scores a model or platform on a specific task. You will see them quoted constantly: a model scores so many per cent on a reasoning benchmark, another tops a coding leaderboard. These numbers are not meaningless, and they are not the answer to your question either.
The thing to understand is that a benchmark measures performance on the benchmark's task, under the benchmark's conditions, on the benchmark's data. Your project is none of those things. A high score on a graduate-level reasoning test tells you very little about how reliably a model will extract the right total from your suppliers' messy, inconsistent invoices. The benchmark task and your task are different tasks.
Three questions turn a benchmark from a headline into useful information: What does this benchmark actually test? How close is the benchmark task to my task? And who produced the benchmark, and on whose model? A benchmark published by a vendor about its own model, chosen by that vendor to feature, has been selected because it flatters the product. Independent benchmarks comparing many models on the same test are more useful precisely because no single vendor chose them.
🔑 A benchmark measures the benchmark: A score tells you how a model performed on a specific test under specific conditions. It becomes useful only when you ask how close that test is to your actual task. Match the benchmark to your use case, or treat the number as noise.
What demonstration videos leave out
A demonstration video shows a platform working. That is exactly the problem. A demo is a curated best case, recorded until it went well, showing the inputs the platform handles cleanly and skipping the ones it does not.
What the demo does not show is the part that matters most for your planning. It does not show the messy input that confuses the tool, the run where the model misreads the instruction, the agent hitting its iteration limit before finishing, or the connector that needed three attempts to authenticate. The demo shows the happy path, and your project will spend much of its life off the happy path. This does not make demos worthless — a demo is a useful way to see an interface and judge whether a platform is worth deeper investigation. It is a starting point for evaluation, not evidence of reliability. The practitioner's question while watching a demo is not "can it do this?" but "what would happen if the input were messier than the one they chose to show?".
Did you know?
A widely shared piece of practitioner wisdom is that the quality of a tool is revealed not by its demo but by its error messages. A demo shows you the tool on its best day. How a tool behaves when something goes wrong — whether it fails clearly or fails silently — tells you far more about living with it day to day.
How to read a capability announcement
New capabilities are announced constantly, and the announcements are written to generate excitement. Reading them critically means asking a few questions the announcement itself will not answer for you.
Is this truly new, or a rebrand of something that already existed? A capability given a new name and a launch event is not always a new capability. What does the announcement itself admit it cannot do? The honest parts are often in the qualifications — "in preview", "for selected users", "in supported regions", "rolling out over the coming months" — which tell you the capability is not yet what the headline implies. And what are practitioners reporting in the first weeks of real use? The gap between an announcement and the experience of people actually using the capability is where the truth lives.
The deeper reason claims mislead: the jagged frontier
Reading sources critically handles the marketing problem. But there is a second, structural reason a claim can mislead you even when it is entirely honest — and it is the single most important thing to understand about designing reliable automation.
Here is something that surprises almost everyone who works with AI seriously. The tasks AI finds easy and the tasks AI finds hard do not match the tasks humans find easy and hard. A model can write a competent first draft of a complex strategy document, a task a person would consider demanding, and in the same session fail to count how many times a particular letter appears in a word, a task a child can do.
This uneven, unpredictable boundary between what AI does well and what it does poorly is often called the jagged frontier. The term comes from a 2023 Harvard Business School field study, Navigating the Jagged Technological Frontier (Dell'Acqua, McFowland, Mollick et al., Working Paper 24-013), which ran a controlled experiment with 758 Boston Consulting Group consultants and found that AI sharply improved their performance on tasks inside the frontier while actually degrading it on tasks that fell just outside — even when the two looked similarly difficult. The word jagged is the important part. The boundary is not a clean line with hard tasks on one side and easy tasks on the other. It is a ragged edge that cuts across our intuitions about difficulty, putting some hard-looking tasks firmly in the capable zone and some easy-looking tasks firmly outside it. A vendor's claim that its tool "handles document processing" may be true for clean documents and false for yours — not because the vendor lied, but because your task sits on a different part of the frontier than the demo did.
This idea is deliberately counter-intuitive, so give it a moment. Most of us carry a strong assumption that "harder for me means harder for the computer" — and the jagged frontier breaks exactly that assumption. Keep returning to the practical consequence: you cannot predict reliability from how hard a task looks, so you have to know the frontier rather than guess it.
Where AI is reliably strong
Current language models are dependable on a recognisable family of tasks — good candidates for automation with lighter oversight, though never with none: synthesising and summarising provided information; drafting structured content from a clear brief; classifying text against clear categories with examples; extracting specified fields from semi-structured documents (with a validation check); translating and converting between formats; generating and explaining code for well-defined problems; and multi-step reasoning when the relevant context is provided. The common thread is that these tasks work with information the model has been given and produce output that fits a recognisable pattern.
Where AI is unreliable despite sounding confident
The harder lesson is the second list, because the failures here are delivered with the same fluent confidence as the successes. The model does not signal that it has crossed onto unreliable ground: precise calculations and arithmetic (a plausible-looking number is not a calculated one); exact factual recall without reference material (the hallucination behaviour from Unit 1, most dangerous on facts that sound plausible); anything requiring a verifiable chain of custody, because the model's fluent synthesis blends sources rather than preserving them; real-time or very recent information beyond its knowledge cutoff; and highly specialised domain knowledge underrepresented in training.
🔑 Confidence is not reliability: A model sounds equally assured whether it is on reliable ground or not. The fluency of an answer tells you nothing about its accuracy. This is why the jagged frontier is dangerous: the failures do not announce themselves.
Capability does not track apparent difficulty. The boundary between what AI does reliably and what it does poorly is jagged, not vertical: some easy-looking tasks (counting letters in a word) fall in the unreliable zone, while some hard-looking tasks (drafting a structured report) fall in the reliable zone. You cannot predict reliability by how hard a task looks — you have to know the frontier.
Why this matters for use-case design
The jagged frontier turns directly into a design rule. The classic and costly mistake in AI automation is to build a workflow that depends on the AI doing reliably something that sits on the unreliable side of the frontier, with no check to catch the failure.
Consider an invoice-processing automation. Extracting the supplier name and the line items sits on the reliable side: that is information pulled from material in front of the model. Verifying that the invoice total is arithmetically correct sits on the unreliable side: that is a calculation, and the model may produce a plausible wrong answer. A workflow that asks the model to extract the figures and trusts it to confirm the total has placed a load-bearing step on the unreliable side of the frontier.
The fix is not to abandon the automation. It is to design with the frontier in mind: let the model do the extraction it is reliable at, and route the arithmetic to a calculation step, or add a validation check that compares the model's total against an independent calculation and flags any mismatch for human review. The automation still works. It simply stops trusting the model with the one task the frontier says it cannot be trusted to do unaided.
Connect this back to your own project. Model literacy from Unit 1 tells you how the engine behaves; the jagged frontier tells you where it behaves unreliably. Look at the steps in your own use case: which of them sit on the unreliable side of the frontier, and does each one have a validation step or human check? A validation step in the right place is the difference between an automation that fails safely and one that fails silently.
Why the frontier shifts
There is one more thing that makes the frontier a living concern rather than a fact to memorise. It moves. With each model generation, some tasks that were unreliable become reliable, and occasionally the reverse. A generation ago, complex multi-step reasoning was firmly on the unreliable side; the reasoning models you met in Unit 1 moved much of it across. The frontier that was true when a project was designed may not be true when it is reviewed.
This is precisely why staying current is not optional housekeeping but core professional practice — the subject of the final lesson. A practitioner who learned the frontier once and stopped looking will, within a year or two, be designing around limitations that no longer exist and trusting capabilities that have quietly changed.
A single vendor claim — "automates invoice processing end to end" — tested against a messy real invoice and mapped onto the jagged frontier. The parts that held (field extraction — reliable side, ticked) sit apart from the parts that broke (total verification — unreliable side, flagged), resolving into a "works / doesn't work / needs a validation step" map for this use case. The lesson: a claim is not true or false in general. It is true on one part of the frontier and false on another, and only testing tells you which.
Test a claim against reality — with your Companion
You have the theory. Now learn the claims-versus-reality skill first-hand, on the tool already in your hand. Your Evidence Companion runs on a platform whose vendor makes claims about it. This activity has two moves: first test a claim against what your Companion actually does, then carry that same scepticism to a claim about a candidate platform for your project — this time verified against documentation rather than by hands-on testing.
Move 1 — test a claim against your Companion's real behaviour.
- Find a claim the vendor makes about your Companion's platform (Claude or Microsoft Copilot) — for example, that it reliably summarises documents, extracts data, or answers questions from your files.
- Put that claim to a deliberately messy test in your Companion: give it a realistic, imperfect input (a document with inconsistent formatting, a question needing a precise figure, a task needing exact recall). Not the clean case a demo would choose.
- Watch where it lands on the frontier. Where did it perform reliably? Where did it sound confident but get something wrong? That gap — between the claim and the messy reality — is the frontier made visible on the tool you are recommending.
Move 2 — apply the scepticism to a candidate platform, via documentation.
Take one capability claim about a platform on your shortlist. Open its marketing page in one tab and its technical documentation or trust/security page in another. Find the claim on the marketing page, then confirm the specifics in the documentation: the marketing page says "secure" or "enterprise-grade"; the documentation says exactly what certification it holds, where data is stored, and under what terms. The documentation version is the one your recommendation can rely on. If a claim appears in marketing but you cannot substantiate it in documentation, that is an unresolved information gap to flag — not a fact to accept.
Optional — go further
Ask your Companion to critique its own limitations: "What kinds of task on this platform are you least reliable at, and where should I add a human check?" Compare its answer to what your messy test actually revealed. A model describing its own frontier is itself a claim to verify — sometimes it is candid, sometimes it is optimistic.
[TEST IT — 15 minutes] Record what your Companion actually did against the claim, and what you verified about a candidate platform.
Claims vs Reality — Your Companion and a Candidate Platform
Test a vendor claim against your Companion's real behaviour, then verify a candidate-platform claim against documentation. This evidences the S25 vendor-evaluation skill on a tool you actually use. Complete it in one go and save as a md file. Save the md file within the same source folder you've given Evidence Companion access.
⏭️ Up next - Lesson 4: You can now test a claim against reality and know where a tool's capability stops. The final lesson turns the shifting frontier into a discipline for staying current — and brings the whole module together into the platform recommendation you confirm with your instructor.