How Intle gates the quality of AI-generated training

Anyone who has watched a language model write training content knows the failure mode. The output reads fluently, looks finished, and is quietly unusable: a "quiz" that turns out to be four multiple-choice questions buried in eleven paragraphs of prose; a flowchart that renders as a wall of raw text; a matching exercise whose answer key marks every learner wrong on every attempt. Fluency is not the same as fitness for teaching. This post is a first-hand walk through the mechanical checks that stand between the model's first draft and the SCORM package you download. They are deliberately narrow — they catch structural and technical failures with high confidence and make no claim to judge whether the content is true. That honest limit is the point I return to at the end.

Four stages, two gate checkpoints

Every generation runs the same pipeline: detection works out what kind of thing you asked for (quiz, module, scenario, compliance and so on), planning turns that into a section-by-section blueprint, an optional structure review lets you approve or adjust that blueprint before any content is written, and generation produces the finished sections. We run the quality gates at two of those stages. The first checkpoint validates the plan before we spend tokens generating from it — a plan that is already the wrong shape can never produce a right-shaped package. The second re-measures the same properties on the real generated output, because a plan can pass and then generation can silently drift away from it: a planned flashcard comes back as a paragraph, a decision loses its response mode. Checking twice is the point.

Structural gates: is this shaped like training?

The first family of gates measures density — the balance of doing versus reading. Each content type carries a minimum interactivity ratio: the proportion of sections that ask the learner to do something rather than just absorb text. The thresholds are tuned per type, because an assessment and a slide deck are not held to the same bar:

• Assessment — 0.8 (four in five sections must be interactive)
• Scenario — 0.7
• Quiz and icebreaker — 0.6
• Compliance, microlearning and interactive session — 0.5
• Module — 0.4
• Presentation — 0.3

Ratio alone is not enough, so the same checkpoint also flags a non-interactive streak of three or more consecutive passive sections (the pattern that makes learners disengage), enforces a visual density floor of roughly 30% of sections carrying a chart, diagram or image, and demands response-mode variety — once a package has four or more questions, it must use at least two answer formats rather than eleven identical multiple-choice stems. There is a pedagogical layer on top: for assessments, modules and compliance content, a Bloom distribution gate fails any package with four or more questions where fewer than 30% reach "apply" or higher, which stops the model settling into a rut of recall-only questions. Some types also have shape requirements — modules and compliance packages must include a flashcard section, compliance must include a comparison table, and briefs that describe a process or an escalation path must include a flowchart. You can see how these thresholds play out in our compliance training examples and across the wider examples gallery.

Content-quality gates: is the content actually usable?

Passing the structural gates means the package is shaped like training. A second family checks whether the individual pieces hold up. Accessibility validators reject media alt-text that is empty, a placeholder like "image", a copy of the section title, or that opens with "image of…" — a floor toward WCAG 1.1.1 Non-text Content rather than a conformance guarantee. Visualisation completeness is where we go furthest: a comparison table needs at least three columns and two rows; a bar chart with every bar the same height carries no information and is rejected; a pie chart cannot have zero or negative segments. The flowchart check is the one I am most attached to, because it is renderer-backed — instead of guessing whether the Mermaid source is valid, we run the actual renderer's parse decision. If the renderer would reject it, the learner would have seen a raw source dump, so we treat that as a hard failure. We also walk the graph for orphan nodes: a step declared but never connected to anything.

Three more validators round it out. Cognitive load caps a context body at 1,200 characters, a question prompt at 50 words, and an answer-option title at 120 characters — long prose gets split or converted to a list rather than shipped as a wall of text. Duplicate detection catches repeated question stems and flashcard fronts, so an eight-question quiz is not secretly asking the same thing three times. And answer-key well-formedness is the quietly critical one: for every question it parses the correct-answer pattern against its response mode — a single-select key must name a real option, a matching key must be well-formed "answer.target" pairs, an ordering key must be a full permutation of the options. A corrupted key is invisible in a preview and marks every learner wrong, so this check is blocking. It is the gate that matters most on our AI assessment generator, where the whole point is a defensible score.

Blocking versus advisory — not every nit is fatal

Early on, every violation killed the whole generation. That was the wrong trade. A leftover alt-text nit, or a topic-overlap heuristic firing on an icebreaker whose section titles legitimately don't echo the brief, would refund the user and deliver nothing — a usable package with a cosmetic flaw is plainly better than no package. So violations are split. Structural failures stay blocking: density, Bloom mix, broken or misleading visualisations, and malformed answer keys. Cosmetic and heuristic findings are advisory: alt-text polish, over-long copy, duplicate stems and fronts, and the token-overlap brief-alignment check. Advisory violations still feed the repair prompt and are recorded for our admin telemetry, but if they are the only thing left standing after repair, the package ships.

One repair pass, then a decision

When the first attempt fails, we do not loop indefinitely. We take the exact list of violations, hand it back to the model verbatim with an instruction to preserve the plan structure while fixing them, and generate once more. After that single repair pass we make a decision: if any blocking violation survives, the generation fails loudly rather than shipping something broken; if only advisory findings remain, the package is delivered with those recorded against it. There is also a graceful-degradation path — if the rich schema call fails outright, a simplified fallback strips the richest visual elements and relaxes the strict density gates so you still get a coherent, if plainer, package rather than an error. These decisions happen in the background; processing time varies with the source and requested output.

What we don't check — yet

Here is the honesty this whole post is built around: none of these gates verify that the content is true. Every check above is structural, technical or heuristic. There is no factual-accuracy or hallucination gate. Concretely, that means we do not check:

• Whether a stated fact is correct — the model can write a confident, well-structured, wrong sentence and every gate will pass it.
• Whether the answer marked "correct" is actually the right answer — the answer-key gate checks the key is well-formed and parseable, not that it points at the truth.
• Whether a regulatory citation is current or accurate — a compliance module can reference the Money Laundering Regulations 2017, or safeguarding guidance such as Keeping Children Safe in Education, and we will not know if the version has moved on.
• Whether the content genuinely matches your brief — brief alignment is a token-overlap heuristic, not a semantic judgement of coverage or emphasis.

Where the gates fit your workflow

In practice the gates provide a structured, interactive first draft that you then open in the editor, fact-check against your own source of truth, and adjust. The output targets SCORM 2004 4th Edition and SCORM 1.2. Dated package-structure and local resume results are published on our LMS compatibility page, but they are not an accessibility-conformance guarantee or named-LMS verification. If you build regulated content, a qualified reviewer must validate every legal, policy, safety and regulatory statement before use.