Review Every Change the Agent Proposes
The back office where analysts check, accept and publish what an AI research agent writes about companies
6 min
Magellan is an AI agent that reads public sources and proposes new values for Liminal’s company profiles. I led the design of the back office where analysts start runs, review every change, handle update requests and publish with a quality check and a full history. It went from a whiteboard to production in about five weeks, and the first analyst to use it rated it 5.5 out of 7.
Industry
Market Intelligence, Cybersecurity & Digital Identity, B2B SaaS
Role
Product Designer (design lead)
Project Duration
6 weeks (Sep–Oct 2026)
Focus
Human-in-the-loop Workflows
Roles & Permissions
Prompt-driven Prototyping
Design in Code
→ The brief
Magellan's output reached analysts as spreadsheets and PDFs, and they wrote their verdicts by hand. Update requests arrived through three channels: a feedback tab, an Asana project and the bug tracker. A single news story, such as an acquisition, often ended up split across two of them.
The design question was not what the agent's output looks like. It was who can do what, in what order, with what evidence, and how every change leaves a trace.
The design question was not what the agent's output looks like. It was who can do what, in what order, with what evidence, and how every change leaves a trace.
→ Research: how analysts already review
Real review files. I read the analysts' review files for one vendor. The justification belongs to the agent and the note belongs to the analyst, analysts add items the agent missed, and a few rejection reasons repeat a lot. The review screen was built on that split.
Request channels. Many requests were “add this company” tasks with only a URL, events that affect several companies, or future effective dates. So the request form accepts companies not yet in the database, several companies per event, a source URL and an effective date.
An audit before anyone saw it. I checked the prototype against the design rules, the code and the real workflow in seven reports. “Start my day” took about 13 clicks across 3 tabs; the high-severity fixes went in the same day.
Request channels. Many requests were “add this company” tasks with only a URL, events that affect several companies, or future effective dates. So the request form accepts companies not yet in the database, several companies per event, a source URL and an effective date.
An audit before anyone saw it. I checked the prototype against the design rules, the code and the real workflow in seven reports. “Start my day” took about 13 clicks across 3 tabs; the high-severity fixes went in the same day.
→ What I designed
Review workspace. Each change is one row: the published value on the left, Magellan's proposal on the right with its justification and sources. Accept and Reject sit below, with keyboard shortcuts and a progress footer.
Reject needs a reason. Reason chips come from the analysts' own procedure.
Run dialog. Four run modes, a required reason, and a choice of where the result lands: a draft for review, or published directly.
Calm dates and a history you can trust. A passed date reads “· passed”, never a red “134 days overdue”. Tier and owner changes are logged with old → new, the reason, who and when.
Reject needs a reason. Reason chips come from the analysts' own procedure.
Run dialog. Four run modes, a required reason, and a choice of where the result lands: a draft for review, or published directly.
Calm dates and a history you can trust. A passed date reads “· passed”, never a red “134 days overdue”. Tier and owner changes are logged with old → new, the reason, who and when.
“Who can do what, in what order, with what evidence?”
→ Decisions, including the ones I reversed
A side-by-side check before every publish, instead of an inline diff, so reviewers see exactly what changes on a live profile.
Urgency as a “Needed by” date, not only a priority flag. Request data showed that dated asks closed in days and undated ones sat for weeks.
Three roles: Analyst, Internal Manager, Admin. My first design let only admins publish; the analyst lead's answers changed it.
A soft launch next to the old process, not an all-at-once switch.
The analyst lead also reversed two of my designs, “ask for more time” and “merge into an existing request”, because both added friction. I shipped that version the same day.
Urgency as a “Needed by” date, not only a priority flag. Request data showed that dated asks closed in days and undated ones sat for weeks.
Three roles: Analyst, Internal Manager, Admin. My first design let only admins publish; the analyst lead's answers changed it.
A soft launch next to the old process, not an all-at-once switch.
The analyst lead also reversed two of my designs, “ask for more time” and “merge into an existing request”, because both added friction. I shipped that version the same day.
→ Working with users seven hours away
The analysts and the developers worked seven hours apart. Instead of waiting for meetings, I sent short Looms with numbered decisions, so the analyst lead could reply “1 yes, 2 no”. Each answer became a prompt round, built and synced the same or the next day.
After the first runs on staging, a five-question survey came back at 5.5 out of 7, with “would use it over the old flow”. The lead then rewrote the team's research procedure around the new flow, and I audited the build against it role by role.
After the first runs on staging, a five-question survey came back at 5.5 out of 7, with “would use it over the old flow”. The lead then rewrote the team's research procedure around the new flow, and I audited the build against it role by role.
→ From prototype to production
About 86 lettered prompts in Claude Design, one change each, checked in the code before the next.
On 3–4 October I ported the prototype to a React app on the real design system, inside a copy of the admin shell. Stakeholders clicked through real components and developers read the code as the spec: 89 commits and 31 shareable versions by 27 October.
Before testers arrived I handed over 13 ticket-ready UI fixes. After launch I wrote the V2 plan as a document before any tickets, and the dev team used it to organise the next epic.
On 3–4 October I ported the prototype to a React app on the real design system, inside a copy of the admin shell. Stakeholders clicked through real components and developers read the code as the spec: 89 commits and 31 shareable versions by 27 October.
Before testers arrived I handed over 13 ticket-ready UI fixes. After launch I wrote the V2 plan as a document before any tickets, and the dev team used it to organise the next epic.
→ Results
V1 went live on 22 October, about five weeks from the whiteboard, with three engineers. The first user rated it 5.5 out of 7 and said they would switch from the old process.
About 86 checked prompts, a React build the developers used as the spec, 13 ticket-ready UI fixes before testing, and a V2 plan the team adopted.
About 86 checked prompts, a React build the developers used as the spec, 13 ticket-ready UI fixes before testing, and a V2 plan the team adopted.
“A great improvement from the first workflow demo.” — Analyst team lead, Liminal
What I learned: influence peaks when a ticket is born, not when a prototype is shown. Cutting inline edit kept V1 shippable, but the first user asked for it right away; next time I would test that cut with users first.
→ What I’d do differently
Test the cuts, not just the features. I cut inline editing to ship V1 on time. The first real user asked for it straight away. Next time I’d check a cut with users before locking it.
Keep today and tomorrow apart from day one. When my prototype drifted behind the live build, I couldn’t show developers the exact journey they were changing. I’d keep a current-state copy next to the proposal.
Ask for decisions early, with a date. A short question with a deadline, sent while the ticket was being written, often moved faster than another prototype round.
Decide how to measure success. We never agreed how to count requests closed in Magellan against the old way, so the impact is still told in quotes, not numbers.
Keep today and tomorrow apart from day one. When my prototype drifted behind the live build, I couldn’t show developers the exact journey they were changing. I’d keep a current-state copy next to the proposal.
Ask for decisions early, with a date. A short question with a deadline, sent while the ticket was being written, often moved faster than another prototype round.
Decide how to measure success. We never agreed how to count requests closed in Magellan against the old way, so the impact is still told in quotes, not numbers.
