A six-week generative AI pilot roadmap

Before week one: agree on the decision

A pilot should reduce uncertainty about a specific workflow. Its outcome is a decision supported by evidence, including the option to stop. The six-week structure below is a planning template, not a promise that procurement, security review, or integration can always fit into six weeks.

Start the clock after the team has a named owner, authorised data access, available reviewers, and a bounded task. If those prerequisites are missing, use the time to resolve them before asking a team to build.

The weekly plan

Week 1 — Observe the work and set a baseline

Watch users complete the task today. Record the full time spent, including checking, rework, and escalation. Agree on examples of acceptable and unacceptable outputs. Deliver a one-page charter with an owner, baseline, audience, exclusions, and approval responsibilities.

Week 2 — Prepare sources and the evaluation set

Choose a manageable, representative collection of authorised documents. Record versions and access rules. Create held-out questions and tasks from real work, including ambiguous requests, absent information, and attempts to access restricted content. Reviewers define what a supported answer looks like before seeing the model output.

Week 3 — Build one complete workflow

Connect input, retrieval, drafting, citations, and human review. Include an explicit “cannot answer from the available sources” path. Keep consequential actions in existing approval systems. Deliver a thin working flow that can be evaluated end to end.

Week 4 — Evaluate and fix critical failures

Run the held-out tasks. Measure source correctness, missing information, unsupported claims, access-control failures, and reviewer effort. Separate recurring failure categories from isolated examples. Fix critical issues before putting the pilot into a real workflow; retain a fresh evaluation set for the next check.

Week 5 — Observe a limited user trial

Give an authorised group clear instructions and a feedback route. Compare total task time and output quality against the original baseline. Count time spent correcting outputs. Look for situations where users accept an answer without checking it, and investigate why.

Week 6 — Make the decision

Compare results with thresholds agreed before the trial. Record technical costs, review effort, adoption, and unresolved risks. Decide whether to stop, revise the workflow, run a longer trial, or fund a limited expansion. Assign ongoing ownership and support before expanding access.

Measure value without inflating it

Track completed work, not just chat volume. A useful scorecard combines task quality, total time including review, user adoption, and critical failure counts. A single average can hide unacceptable failures.

Use comparable tasks and record differences in their difficulty. Small pilot samples carry uncertainty: document what was not tested, seasonal conditions that were absent, and which teams may have different requirements.

Write the decision gate before the demo

• Stop: required access controls cannot be enforced, critical errors remain unresolved, or the workflow offers no useful improvement.
• Revise: a narrower task, better sources, or a different user experience may address specific failures.
• Expand carefully: agreed quality and value thresholds are met, ownership is funded, and monitoring and rollback are ready.

The gate should name who decides and what evidence they need. A polished demo is useful for communication; the evaluation record is what supports the investment decision.

Before planning, assess your team’s readiness. For broader governance guidance, consult the NIST AI Risk Management Framework.