AI Agent Underwriting Product
- Client
- Samsung Fire & Marine Insurance
- Date
- 2024–2025
- Role
- Product & Strategy Lead → PMO
- Team
- 38 people · Consulting, Design, Engineering, AI & Architecture
Context
Korea's insurance market is contracting along with its population. Growth has to come from overseas — and overseas expansion requires the company's most experienced underwriters, the same people holding up domestic operations. That was the real constraint.
Underwriting expertise lived in individuals: years of pattern recognition built case by case, written down nowhere. Before anyone could be redeployed, that judgment had to become something the organization owned rather than something specific people carried.
There was no product to improve. Underwriting ran on documents and judgment, and whatever we built would be the first version of something that had never existed here.
The Problem
Underwriters didn't distrust the AI because it was wrong. They distrusted it because they couldn't see why it was right.
And every fix made the product worse. Showing full reasoning made the interface heavier. Narrowing what the agent could decide made it less useful. The more we did to earn trust, the less time the tool saved — and a tool that doesn't save time doesn't get used, however much people trust it.
What I Decided
1. Surface keywords, not summaries or confidence scores.
I spent time watching underwriters work before specifying anything. They skim large volumes of documents, and they skim for keywords — that was already the unit of thought in their heads. Summaries would have asked them to trust a compression they couldn't inspect. Confidence scores would have asked them to trust a number they had no way to calibrate. Keywords asked them to trust nothing: they could see the signal and judge it themselves.
2. Aggregate five sources into one view, with evidence one click away.
Risk signals were scattered across notes from the policy design stage, underwriters' own review text from comparable industries, third-party survey reports, incident reports from prior claims on renewals, and external news on the client and its sector. Underwriters were opening five systems to assemble one picture. I capped it at five keywords per source, ranked on historical outcomes. Uncapped, the aggregation would have recreated the noise problem it was meant to solve — thirty signals on one screen is not a summary. Each keyword drilled through to its source document, so evidence stayed available without being permanently on screen.
3. I argued my own team down on automation scope.
The team wanted the agent handling underwriting end to end at launch — technically achievable, and the client would have accepted it. I pushed for the opposite: the agent proposes, underwriters review and approve, and the boundary widens only as output quality proves out. The argument was sequencing. A phase two was already planned. If we shipped full automation and it misfired once, we would spend phase two rebuilding trust instead of extending capability. Shipping narrow bought us the right to expand later. We launched with 3 of 8 underwriting stages fully automated and ended at 6.
4. Dwell time was the metric I watched to decide when to expand.
Loss ratio and new contract rate were the outcomes that mattered, but both lag by months — too slow to govern a rollout. So I tracked how long underwriters lingered in each stage of the system. Long dwell meant friction: something the agent surfaced wasn't trusted, or wasn't legible, or wasn't enough. When dwell dropped and stayed down on a given stage, that stage was a candidate for full automation, and we made the call jointly with the client.
What Didn't Work
We scoped nine initiatives for phase one and shipped five. The infrastructure required before any of them could run was substantially larger than our estimate, and we discovered that after the scope was committed. Four initiatives moved to phase two. The estimate was mine to get right and I got it wrong — I sized the AI work and underweighted everything underneath it.
The Outcome
Underwriting turnaround dropped 40% and loss ratios improved 18%. But the outcome that mattered most was variance.
Underwriting quality had always tracked seniority — the same case could be priced very differently depending on who reviewed it. With risk signals surfaced consistently across every review, less experienced underwriters began making calls closer to those of the most senior ones. The floor came up.
That was the point. Expertise locked in individuals became something the organization could rely on, which is what made redeploying senior underwriters possible. It also explains the loss ratio: the losses were never coming from the best people.
What I'd Do Differently
I'd design the data layer for the model, not for the database. This was my first time building on an LLM, and I specified table structures the way I always had — normalized, application-oriented, optimized for retrieval by a system that knows exactly what it's asking for. A model doesn't ask that way. Getting the right context into a prompt meant assembling it across tables that weren't shaped for it, which made every retrieval path more fragile than it needed to be. Next time I'd start from the question of what a prompt needs to receive, and work backward into the schema.