Lesson 3 - Build Your Platform Evaluation Matrix

Module 5, Unit 2 | Lesson 3 of 3

By the end of this lesson, you will be able to:

• Convert your L2.2 shortlist into a scored Platform Evaluation Matrix (K25, S25)
• Separate hard constraints from weighted preferences before making a recommendation (K8, S27)
• Score each candidate with evidence from current sources, not opinion or brand familiarity (S25, K25)
• Write a platform recommendation that names the route, the evidence, the trade-off and the approval questions (S27)
• Upgrade your Commit Log Agent so it can review your matrix logic and evidence quality (S27)

L2.1 gave you the route. L2.2 gave you the shortlist. L2.3 turns that work into a technical decision pack.

You are going to build two evidence artefacts:

module-5/unit-2/l2_3_platform_evaluation_matrix.xlsx
module-5/unit-2/l2_3_platform_recommendation.md

If you do not want to use a spreadsheet, you can build the matrix as:

module-5/unit-2/l2_3_platform_evaluation_matrix.md

This is not a repeat of the prioritisation or business-case work from earlier modules. Here, the decision is technical: which platform route is most suitable for the AI component you are designing, given the evidence from L2.1 and L2.2?

Why the Matrix Exists

A platform recommendation is easy to write badly. It can become a list of benefits, a favourite provider, or a vague statement such as "this platform is powerful and scalable."

A strong recommendation is different. It shows:

• which candidates were evaluated
• which constraints were non-negotiable
• which criteria mattered most for the project
• what evidence supported each score
• why the recommended option fits the route from L2.1
• what trade-off is being accepted
• what still needs approval before implementation

The matrix is not there to make the decision for you. It is there to make your reasoning visible.

Coach Cora

A matrix should discipline your judgement, not replace it. If one option wins by arithmetic but fails a hard constraint, the answer is not to ignore the constraint. The answer is to explain why the option cannot be recommended yet.

The diagram below shows how candidates move through the hard constraints gate before reaching weighted scoring and the final recommendation.

Build Part 1 - Choose the Candidates

Open your L2.2 file:

module-5/unit-2/l2_2_platform_shortlist.md

Open module-5/unit-2/l2_1_modality_decision.json now and keep it alongside the matrix — your scoring should reflect the route and constraints your agent identified.

Select two or three candidates for the matrix.

Use these rules:

• Include candidates marked keep.
• Include candidates marked needs more evidence only if the missing evidence can be checked in this lesson.
• Do not include a rejected candidate unless you want to show why it fails a hard constraint.
• Do not add a new provider just because it is well known. Add it only if it fits the L2.1 route and you can gather evidence.

At the top of your matrix, add:

## Candidates evaluated
| Candidate | Platform route | Link to L2.2 shortlist row | Included because |
|---|---|---|---|
|  |  |  |  |

This table prevents the matrix from drifting away from the shortlist. The reader should be able to trace every candidate back to L2.2.

Build Part 2 - Mark Hard Constraints

Before scoring anything, identify the hard constraints.

A hard constraint is a rule or requirement that cannot be averaged away. If a candidate fails it, a high score elsewhere should not rescue it.

Hard constraints often include:

• data must stay in an approved environment
• the supplier or platform route must be allowed by the organisation
• the service must support the file type or input route from L2.1
• low-confidence or sensitive cases must go to human review
• the platform must provide logs or evidence suitable for review
• the cost model must be viable at the expected usage level

Add this table:

## Hard constraints
| Constraint | Why it matters | How I checked it | Candidate A | Candidate B | Candidate C |
|---|---|---|---|---|---|
|  |  |  | pass / fail / unknown | pass / fail / unknown | pass / fail / unknown |

Use unknown honestly. Unknown is not a failure, but it is not approval either. It means the recommendation must pause, escalate or carry the question forward.

Curious Cat

In the 1960s, US Secretary of Defense Robert McNamara ran the Vietnam War using detailed scoring models — tracking body counts, aircraft missions, supply tonnage. The numbers looked excellent. The war was being lost. Sociologist Daniel Yankelovich named the resulting trap McNamara's Fallacy: measure what is easy to measure, treat it as important, ignore what cannot be measured, and ultimately make decisions that look rigorous but miss what actually matters. Evaluation matrices have the same failure mode. A candidate can score well on every criterion you remembered to include and still fail on the one you forgot — or the one you marked "unknown" and quietly left behind.

Build Part 3 - Define the Criteria

Use the criteria below unless your project needs a small adjustment.

You may add one project-specific criterion. Examples:

• regional data residency
• accessibility
• audit logging
• compatibility with an existing cloud environment
• procurement or licensing constraints
• support for batch processing

Build Part 4 - Score With Evidence

Use a 1-5 score for each criterion:

Every score must include an evidence note. Do not write:

4 - good

Write:

4 - Official documentation confirms regional processing is available, but internal approval still needs confirmation.

Use this matrix structure:

## Scored matrix
| Criterion | Weight | Candidate A score and evidence | Candidate B score and evidence | Candidate C score and evidence |
|---|---|---|---|---|
| Route fit |  |  |  |  |
| Task performance evidence |  |  |  |  |
| Data handling and compliance |  |  |  |  |
| Cost and usage limits |  |  |  |  |
| Integration effort |  |  |  |  |
| Human review and control |  |  |  |  |
| Maintainability and lock-in |  |  |  |  |
| **Total** |  |  |  |  |

Assign a weight to every criterion using this scale:

• 3 = critical for this project
• 2 = important
• 1 = useful but not decisive

Then calculate:

weighted score = score x weight

For example: if Route fit scores 4 with weight 3, the weighted score is 12. If Data handling scores 3 with weight 3, the weighted score is 9. Two criteria total: 21 weighted points.

The score helps you compare. It does not override a hard constraint.

Build Part 5 - Write the Recommendation

Create this file:

module-5/unit-2/l2_3_platform_recommendation.md

Use this structure:

# L2.3 Platform Recommendation

## Recommended route
- Recommended platform route:
- Candidate or provider name:
- Link to L2.1 modality decision:
- Link to L2.2 shortlist:
- Link to L2.3 matrix:

## Decision summary
Write four to six sentences explaining the recommendation in plain English.

## Evidence that supports the recommendation
- [evidence point]
- [evidence point]
- [evidence point]

## Hard constraints
Explain which hard constraints passed, failed or remain unknown.

## Trade-off accepted
Name one significant trade-off you are accepting and explain why it is acceptable at this stage.

## Risks and open questions
List what must be checked before this becomes a real implementation, procurement or workplace decision.

## Human review and approval
Name who should review the recommendation next: coach, instructor, line manager, technical lead, data protection contact, procurement contact or another appropriate role.

The recommendation should sound like an engineering decision, not a sales pitch. It should be clear enough for a non-specialist stakeholder to follow, and specific enough for a technical reviewer to challenge.

Build Part 6 - Add a Sensitivity Check

Before you finish, test whether your recommendation is fragile.

Add this short section to your recommendation:

## Sensitivity check
- If compliance became more important, would the recommendation change?
- If expected usage doubled, would the recommendation change?
- If OCR or multimodal quality was weaker than expected, would the recommendation change?

Answer each question in one or two sentences. This is where strong judgement shows. You are not pretending the decision is perfect. You are showing what would make you revisit it.

Challenge Chase

Add a reversal point to your recommendation: one clear condition that would make you change your mind. For example, "If internal approval for this cloud route is not granted, the recommendation changes to the approved OCR-first route." This makes the recommendation more honest and more useful.

Upgrade Your Commit Log Agent

Open your commit-log-agent.md file. Add this reusable L2.3 prompt:

## L2.3 - Platform Evaluation Matrix Review

Use the Commit Log Agent rules above. Help me review and document my L2.3 Platform Evaluation Matrix and recommendation.

Context:
- L2.1 modality decision file or link:
- L2.2 shortlist file or link:
- Matrix file or link:
- Recommendation file or link:
- Candidates evaluated:
- Hard constraints:
- Criteria and weights used:
- Highest-scoring option:
- Recommended option:
- Trade-off accepted:
- Reversal point:
- Sources checked:
- Evidence I can safely share:
- Evidence I must keep private:

Please draft:
1. A short evidence review: does the recommendation follow from the matrix, hard constraints and sensitivity check?
2. A concise Commit Log entry for this lesson.
3. A GitHub-ready README note.
4. A suggested commit message.
5. Three questions I should discuss with my coach, instructor or line manager before implementation.

Before drafting, ask me up to three questions if any important detail is missing. Do not invent scores, sources, links, policies, prices, platform capabilities, workplace systems, approvals or confidential details.

Use the prompt after your matrix, recommendation and sensitivity check are complete. Review the agent output yourself before adding it to your evidence.

Add It to Your AI Projects Repository

If it is safe to share, add these cleaned files to your AI Projects repository:

module-5/unit-2/l2_3_platform_evaluation_matrix.xlsx
module-5/unit-2/l2_3_platform_recommendation.md

If you used Markdown for the matrix, add:

module-5/unit-2/l2_3_platform_evaluation_matrix.md
module-5/unit-2/l2_3_platform_recommendation.md

Add this README note:

### L2.3 - Platform Evaluation Matrix

I built a Platform Evaluation Matrix using my L2.1 modality decision and L2.2 platform shortlist. The matrix marks hard constraints, scores candidate routes with evidence and supports a written recommendation with a trade-off, sensitivity check and open approval questions.

Use one of these routes:

• GitHub browser route: upload the cleaned files and use the commit message add L2.3 platform evaluation.
• Local Git route: save the files inside your local repository, then run:

git add module-5/unit-2/l2_3_platform_evaluation_matrix.xlsx
git add module-5/unit-2/l2_3_platform_recommendation.md
git commit -m "add L2.3 platform evaluation"
git push

If you used a Markdown matrix instead of a spreadsheet, replace the .xlsx file in the command with:

git add module-5/unit-2/l2_3_platform_evaluation_matrix.md

If your matrix contains private workplace information, supplier terms or internal approval notes, publish a cleaned version and keep the detailed version private.

Review Before You Keep or Publish It

Before you add anything to GitHub, your portfolio or apprenticeship evidence, check:

• The matrix uses candidates from your L2.2 shortlist.
• The criteria connect to your L2.1 route and workplace constraints.
• Hard constraints are marked before the final recommendation.
• Every score has an evidence note.
• Unknowns are not hidden.
• The recommendation names a trade-off and a reversal point.
• Confidential information has been removed from the public version.
• The Commit Log Agent entry is accurate and reviewed by you.

Checklist

• My evaluation matrix is created (as .xlsx or .md) and uses the L2.1 decision and L2.2 shortlist as inputs
• Hard constraints are marked before any scoring begins
• At least two realistic candidates are scored with evidence notes — no bare numbers
• My platform recommendation is written with a trade-off, sensitivity check and reversal point
• I upgraded my Commit Log Agent with the L2.3 review prompt
• Cleaned evidence is committed to my AI Projects repository, or I have recorded why it cannot be shared publicly yet

Quick Check

KSB evidence focus

• K8 - The capabilities, risks and implications of on-premise, cloud-based and third-party solutions. Your matrix makes those implications visible through hard constraints, data handling, cost, integration, human review and maintainability evidence.

• K25 - Approaches to maintaining current knowledge of AI platforms and sector trends. Your evaluation depends on current source evidence, not assumptions about provider capability or popularity.

• S25 - Keep up to date with AI, automation and technology, including methods to evaluate vendor and supplier solutions. The Platform Evaluation Matrix is a practical method for comparing vendor or supplier routes using documented criteria, sources and evidence-backed scoring.

• S27 - Apply technical understanding to help align business needs with technical capabilities. Your recommendation translates technical evidence into a decision stakeholders can review, challenge and approve before implementation.

Up next: Unit 3 uses your platform decision to move into implementation: making API calls, handling service errors, controlling cost and connecting tools safely.