Lesson 4: Beyond Text — Multimodal AI

By the end of this lesson, you will be able to:

• Identify automation use cases enabled by multimodal capabilities that text-only processing cannot support (K9)
• Explain transcription, image analysis, and image and video generation, and recognise which are applied and which are awareness-level for this pathway (K9)
• Treat multimodal capability as a model and platform evaluation criterion where your project's data warrants it (K9, S25)
• Decide what your Evidence Companion can ingest, and test it by turning a real, consented spoken conversation into a drafted evidence entry (K9, B6)

Beyond text

The previous three lessons treated the model as something that reads text and writes text. That is where most no-code automation lives, but the models underneath increasingly handle far more than text — and that wider capability opens automation use cases that text alone cannot reach.

Multimodal means a model can work with more than one type of data: text, yes, but also audio, images, and increasingly video. For a practitioner, the practical question is simple. If your project's information currently exists as something other than digital text, multimodal capability may turn a process you assumed could not be automated into one that can.

Transcription

Transcription is the conversion of spoken audio into text. A model with transcription capability receives an audio file and returns structured text that the rest of a workflow can then process.

This unlocks any process where valuable information currently exists only as audio. A construction estimator who records site notes verbally rather than typing them. A sales team whose call recordings contain details that never make it into the CRM. A team running meetings whose decisions live only in the recording. In each case, transcription turns audio into text, and once it is text, every text-based automation step becomes available: summarise it, extract action items, classify it, file it, route it.

You are probably already using it

You do not have to imagine this capability — most workplaces already have it switched on. The conferencing tools teams use every day now transcribe meetings as standard.

• Microsoft Teams produces a live transcript and an AI-generated recap of decisions and action items.
• Google Meet offers transcription and "take notes for me" inside Workspace.
• Zoom records, transcribes, and summarises calls through its AI Companion. In each case the meeting audio is converted to text and then summarised — exactly the audio-to-text-to-automation pattern described above, packaged into a product you already have.

A newer category of dedicated meeting tools has made this the whole point rather than a side feature. Granola is the most talked-about example right now: it listens to a meeting, transcribes it, and turns your rough notes into a clean, structured summary afterwards, working across whatever calls you join rather than living inside one conferencing platform. Tools like this are popular precisely because they remove the manual note-taking step entirely — the transcription and the summary happen in the background while you concentrate on the conversation.

A word of caution

Convenient as these tools are, treat third-party transcription at work with care. A standalone notetaker like Granola sends your meeting audio to an outside service, which may store it, process it, or use it to improve its models. That can mean confidential discussions, client data, or colleagues' words leaving your organisation's control — sometimes without everyone on the call knowing they are being recorded. Before using any transcription tool at work, check that it is approved by your organisation, that participants are told recording is happening, and that the vendor's data terms are compatible with the confidentiality your meetings require. A sanctioned tool inside your existing suite (Teams, Google Meet, Zoom) usually keeps data within a boundary your organisation already governs; a personal tool bolted on top often does not.

Transcription is easy to underestimate. A huge amount of workplace information is spoken before it is ever written down — and much of it is never written down at all. Think of it as the bridge from "spoken and lost" to "structured and usable". Before moving on, ask yourself where spoken information disappears in your own project or workplace.

Image analysis

Image analysis is the ability of a multimodal model to receive an image and reason about its contents. This is different from simply storing or displaying an image; the model can describe what is in it, extract information from it, and make judgements about it.

The use cases cluster around situations where a human currently looks at something and makes a judgement. Automated document processing where the information sits in a scanned image rather than as extractable digital text. Quality control where a visual output is checked against a standard. An HR team receiving application documents as photographs. An operations team monitoring equipment through camera images. A marketing team pulling catalogue data from product photographs.

The most practically important case for many learners is the scanned document. A workflow that receives invoices, application forms, or contracts will often receive some of them as scanned images or photographs rather than as digital text. A platform running a vision-capable model can read those cases directly, without a separate optical character recognition step bolted onto the workflow.

The capability is in the models you already use

Image analysis is not a specialist feature you have to go looking for. Nearly all current frontier models accept an image alongside text and reason about it:

• Anthropic Claude — the Opus / Sonnet / Haiku 4.x line is all vision-capable.
• OpenAI — the GPT-5 family and GPT-4o accept images.
• Google Gemini — 2.x (Pro / Flash) is natively multimodal, image input included.

That means the capability is available through almost any platform whose underlying model is current: you can photograph a receipt, screenshot an error message, or snap a whiteboard after a meeting, hand it to the model, and get structured text back. The practical question for your project is therefore rarely "can a model do this?" and almost always "does the platform I am evaluating run a model that can, and have I confirmed it?"

🔑 Multimodal as an evaluation criterion: If your workflow will process documents that are sometimes scanned images rather than digital text, a platform that supports vision-capable models handles those cases without a separate integration. That makes multimodal capability a real evaluation question, not a curiosity.

[TRY IT NOW — 3 minutes]

Open any LLM available to you (Claude, ChatGPT, Gemini, or another). Find any image on your device that contains text or information you would normally read with your eyes: a screenshot of a table, a photo of a printed document, a diagram, a receipt. Upload it and ask the model to extract the information into a structured format, for example a list or a table.

Watch it read the image and return structured text. That single step, image to structured text, is the multimodal capability that turns "a human has to look at this" into "a workflow can process this". Now imagine that step running automatically on every scanned document your project receives, rather than you doing it by hand.

If your image extracted cleanly — good. Now ask the harder question: what would happen to your workflow if one in ten images extracted imperfectly? Any automation that relies on image extraction needs a validation step, just as any automation that relies on text generation does. Keep that in mind when you decide what your Evidence Companion should take in.

Image and video generation

Two further capabilities are worth knowing about, though they sit at awareness level for this pathway rather than applied level.

Image generation is the creation of new images from text descriptions. It is less directly relevant to most operational automation, but it matters for marketing and communications work: generating illustration concepts, producing draft visual assets for review, or creating consistent imagery for content workflows. Several tools are in common use:

• OpenAI — image generation built into ChatGPT, powered by GPT Image (its gpt-image-1 model) and DALL·E 3, strong for conversational, iterative creation.
• Google — Imagen and Gemini's native image generation and editing, including the model nicknamed Nano Banana, well regarded for editing existing images.
• Open-weight models — FLUX (Black Forest Labs) and Stable Diffusion, run by teams that want control over the model.

Optional — how image generation actually works

You do not need this to use these tools, but it is a genuinely clever idea. Most of them work by diffusion. During training, the model is shown millions of images that have had random visual noise added step by step until each is pure static, and it learns to predict and undo that noise. Once trained, you can hand it a canvas of pure noise and it will "denoise" it in stages into a coherent image. Your text prompt steers that denoising: the model has learned to associate words with visual patterns, so "a red fox in snow" nudges each step towards an image that matches your words. You are effectively watching it sculpt a picture out of noise, guided by your description — much as an LLM predicts the next token for text, a diffusion model predicts its way from noise to pixels.

Video generation is the creation of moving footage from a text prompt or a still image, and it is the fastest-moving corner of this space. It is awareness-level here because the quality, cost, and licensing terms are still settling, but it is worth knowing the leading tools:

• Google Veo — widely regarded as top-tier, with native audio, available through Gemini.
• OpenAI Sora — text-to-video with strong prompt adherence, available through ChatGPT.
• Runway — popular with creators and filmmakers for its editing controls.
• Kling, Luma Dream Machine, and Pika — further strong options in active use.

A related capability, video analysis, is the emerging ability to reason about video content: summarising recordings, identifying events, extracting information. It is worth tracking for use cases involving recorded demonstrations, training materials, or safety footage. For now, treat both video generation and video analysis as capabilities to watch rather than ones to build a project around.

Did you know?

The same underlying model often handles several modalities at once. A single modern model can read your text, look at your image, and listen to your audio within one request. For example, in Gemini you can attach a photo of a paper receipt and a voice note saying "this was Tuesday's client lunch, split it across the two projects" in the same message, and the one model reads the image, listens to the audio, and returns a structured expense entry — no separate OCR tool, transcription tool, and text model wired together. The "multi" in multimodal is increasingly literal — not separate specialist models bolted together, but one model that handles several data types natively.

What your Evidence Companion can ingest

In Lesson 1 you built your Evidence Companion and wrote its system prompt; in Lesson 2 you stress-tested its guardrail against fabrication; in Lesson 3 you chose the model that reasons inside it. This lesson decides the third thing: what it can take in. A Companion that only accepts text you type out by hand is far less useful than one you can hand a document, a screenshot, or — the richest source of all — a recording of you actually talking about your work.

Most of the evidence of what you did at work is spoken long before it is ever written: in a stand-up, in a handover, and above all in your one-to-ones with your instructor or apprenticeship manager. That conversation is precisely where you explain what you changed, why, and what happened — the raw material of an evidence entry. Transcription is the bridge that turns it from "spoken and lost" into something your Companion can shape.

[BUILD IT — 20 minutes] Feed your Companion a real conversation

This is the module's most authentic Companion step. You will turn a spoken conversation about your work into a drafted evidence entry — practising the exact workflow you will reuse for the rest of the apprenticeship.

1. Get consent first. Before your next one-to-one with your instructor or apprenticeship manager, ask their permission to record and transcribe it, and explain why (you are using it to draft your own evidence). Recording someone without telling them is never acceptable, however convenient. You do not need a fresh recording for this — if you already have a past session where everyone gave permission at the time, you can reuse that transcript instead.
2. Use an approved tool. Capture the transcript with a tool your organisation sanctions — the Teams, Google Meet, or Zoom transcript, or your organisation's approved notetaker — not a personal consumer app bolted on top. This is the "word of caution" above, put into practice.
3. Match what you paste to your Companion's tier. This is where your Lesson 3 policy check pays off, and the right amount of redaction depends on the tier you chose. If your Companion runs on a consumer chat tier, strip anything sensitive before it goes near the model — names of third parties, customer details, anything commercially confidential — because that tier may train on your input. If it runs on a sanctioned business or enterprise tier — for example Microsoft 365 Copilot with Enterprise Data Protection, or a Claude business tier — your data stays inside your organisation's governed boundary and is not used to train the foundation models, so blanket redaction is not required. Even then, the tier is not a blank cheque: only include what your organisation's policy permits for that tool, and remember the other person's consent covered being recorded, not having every detail they shared processed further. When unsure, redact.
4. Hand it to your Companion. Paste the cleaned transcript in and ask it to draft one evidence entry from what you said you did. Read the draft critically: it should structure and sharpen your account, never invent achievements you did not describe.

If you cannot arrange a recorded 1:1 right now, do a lighter version: record a two-minute voice note describing something you changed at work this week, transcribe it, and run that through instead.

📺 Optional — going deeper: If you would like a fuller picture of how multimodal AI evolved and works under the hood, the short video below explains how a single model comes to process and generate text, images, and video together. It is enrichment, not required viewing — feel free to skip it and read on.

💬 Reflection prompt: Where in your own work does valuable information exist as something other than typed text — spoken in meetings, captured in photos, buried in scanned documents? Pick the one source that, if your Companion could ingest it reliably, would save you the most effort, and note what would have to be true (consent, an approved tool, an acceptable data tier) before you could use it.

Quick Check

⏭️ Up next - Unit 2: You have finished the model layer — how models work, which one to choose, and what they can take in. Unit 2 turns to the platform layer that wraps the model: the four families of AI and automation platform, how to read any one of them against a consistent set of questions, and how their capabilities differ in practice.

Unit 1: How the knowledge in this unit maps to the apprenticeship