PM Experiment Copilot

Experimentation · AI copilot · Product analytics

An AI assistant that helps product managers generate hypotheses, prioritize experiments, define success metrics, and analyze results — making experimentation faster and more structured without requiring deep analytics expertise.

Problem and impact

Most PM tools optimize for the parts of experimentation that are already easy — tracking status, storing results. The part that actually determines whether a team ships the right thing is upstream of that: turning "onboarding completion is falling" into a problem statement that isn't just a feature idea wearing a disguise. That step usually happens in someone's head, gets skipped under deadline pressure, and the cost shows up months later as a team re-litigating a decision nobody can explain, or shipping a fix for a problem they never actually verified.

My role and decisions

I designed the product end to end: the core object model, the workflow, and the AI behavior spec. The decision I'd defend hardest is making the Decision Workspace — not the experiment — the unit of work. Every problem gets one workspace holding the problem statement, the evidence, every hypothesis considered including the rejected ones with reasons, the experiment design, and the final decision record. I also decided the AI should classify every piece of evidence a PM enters as fact, assumption, opinion, or unknown, rather than treating all input as equally true. That one rule does more to prevent bad decisions than any hypothesis-generation feature.

The AI's job is to challenge, not generate.

Solution and workflow

The workspace moves through a fixed lifecycle — Draft → Problem Defined → Hypotheses → Experiment Planned → Running → Results Ready → Decision Made → Archived — so status is always legible without a status meeting. Five screens carry the whole flow: a dashboard, the workspace itself, a hypothesis board for scoring and comparing options side by side, an experiment plan the AI reviews before launch, and a results screen for interpreting what happened. Rejected hypotheses stay visible with their rejection reason, so teams stop re-pitching the same idea every quarter.

AI design

The AI's job is to challenge, not generate. A weak version of this product hands over three hypotheses and calls it done. Mine is built to catch the PM smuggling an assumption into a problem statement — "onboarding is too long" when the evidence only shows where people drop off, not why — and say so before a sprint gets spent testing the wrong thing. Rules, not the model, own the hard boundaries: the AI can suggest a priority ranking for hypotheses, but the PM makes the final call; it can flag a weak experiment design (no primary metric, no guardrails), but it never declares an experiment successful on its own. The model proposes and questions; the human still owns every irreversible decision — what to test, what to ship, what to kill.

Trade-offs and metrics

I explicitly kept out live analytics integrations, auto-launched A/B tests, and Jira/Linear sync — all the things that make a demo look more finished than the underlying idea is. Results get entered manually for now, because the part worth proving isn't a data pipe, it's whether the interpretation step actually changes what a PM decides. Since this is a portfolio build rather than a shipped product with real usage, I'm not claiming numbers I don't have. What I'd measure in a real deployment: does time-to-decision shrink, do decision records actually get reread later (a proxy for whether they're trusted), and does the rejected-hypothesis log measurably cut down repeat pitches.

What I would bring to your team

This project is the clearest proof I have of one specific instinct: AI features are safest when they're built to disagree with the user, not just assist them. That's a harder design problem than a clean prompt-to-output wrapper, and it's the one I default to whenever a model's output is about to become someone's actual decision.

Next project
Payroll AI