How should an organisation evaluate an AI authoring tool?

Evaluate an AI authoring tool with the same realistic source, audience and learner task in every candidate. Score the accuracy and edit effort of the resulting draft, inspect activity and accessibility behaviour, export to the receiving LMS, and verify data handling, rights and actual cost in writing. Label every finding tested by your team, vendor-documented or unknown. A feature list or polished sample alone cannot answer whether the tool fits your workflow.

Set one test brief before opening a demo

Choose a source your team has permission to use, with one rule that must be quoted accurately, one decision learners must practise and one scored response. Decide the target audience, review owner, LMS edition, language and accessibility needs before testing. Give each vendor the same inputs and the same time budget. Keep confidential material out of a trial until its data terms and access controls are approved.

Measure five outcomes, not the number of buttons

1. Draft fidelity and review time: record omitted, invented or misquoted source claims; time how long a qualified editor needs to make the content safe and useful.
2. Learning interaction: count distinct working activities, inspect answer and feedback quality, then try them as a learner rather than accepting a catalogue badge.
3. Author control and portability: test editing a section, changing the plan, exporting the final artifact and retrieving your source and course data under the contract.
4. Delivery and reporting: import the exact ZIP into your LMS, test score, completion and resume, and separately run a hosted session if that matters. A vendor's format claim is not a passed institution-specific test.
5. Governance and cost: ask for AI provider and retention terms, security review, rights and data export, participant limits, generation and edit allowances, renewal terms, support terms and taxes applicable to your purchase.

Keep an evidence column beside every score

Use a simple 0–3 scale for each criterion: 0 fails a required task, 1 works with major manual repair, 2 works with bounded repair, and 3 meets the agreed acceptance test. Store the artifact, evaluator, date and notes. A vendor documentation page can establish a claimed feature or supported format, but cannot substitute for your own LMS result. Mark untested cells unknown rather than giving them a middle score. Weight mandatory requirements before adding scores so a failed privacy or LMS gate cannot be hidden by attractive media.

A fair Intle example of the same method

Intle's public examples let you inspect stored output and request a SCORM 2004 sample. Their capture dates and subject-matter approval are not independently recorded, so they are examples of possible output, not measured average quality. The pricing comparison shows current author and monthly capacity; each plan uses the same creator toolkit, with differing allowances. The LMS matrix separates a historical SCORM Cloud run and scoped Blackboard pilot from pending named-platform validation. Apply the same evidence standard to Intle as to any other candidate.

How to compare other vendors

For each competitor, record the exact product edition, pricing date and a link to its current primary documentation for SCORM formats, editing, branching, reporting, source handling and export rights. Mark those cells vendor-documented until your own team runs the workflow. Do not infer LMS compatibility from a logo, compare a trial plan with another vendor's enterprise plan, or publish claims about an untested product. A buyer can then see the strengths and unknowns of every option, including Intle.