Background
Hospitals manage safety incidents every day, from medication errors to workplace violence. The people responsible for reviewing these reports—called incident owners—investigate what happened and look for ways to prevent similar events.
My initial work with an Upstate New York health system focused on helping employees report incidents using AI. I also designed a North Star Copilot for the safety teams reviewing those reports, extending the product vision from better reporting to AI-assisted analysis and review.
The design centered on a question: How much should an agent do independently, and where does the user need visibility or control?
Impact
Secured client buy-in by setting the vision for AI-powered incident management
The North Star was a major reason the client committed to the initial intake work. I made AI’s potential tangible: how it could support investigations, what it could do independently, and where people would retain control. Demonstrating both the value and the interaction model built confidence in the broader product vision.
Related case study: Finding the Right Place for AI in Hospital Safety
Personalize the assistance, explain the choices
A safety team member starts with a question: “Where is workplace violence increasing?”
Copilot personalizes the analysis using the user’s previous behavior. Here, it selects a 90-day timeframe based on how they typically review safety trends and explains that choice in chat. The user can adjust it at any point. When more information is needed, a compact form helps establish the scope.
The conversation shows progress and important decisions. A dedicated canvas presents the completed analysis, giving the user a clear place to examine the result.
Design principle: Use context to reduce setup, and explain consequential choices. Personalization should give users a useful starting point they can understand and change.
Build on previous analysis
One finding often leads to another question. Users can ask for a breakdown by work shift, adjust the timeframe, or explore a related comparison.
Each session keeps the conversation and its analyses together. Users can switch between results, pin useful outputs, and return to previous work without starting over.
Design principle: Preserve context across questions. Keep related analyses and their conversation connected so users can build on previous work.
Build trust through inspectable evidence
Building trust in an AI-generated finding starts with being able to inspect the evidence behind it.
A source preview connects each analysis to the underlying incident reports. Users can open the relevant records in the existing incident-management application, then return to the analysis without losing their place.
This keeps supporting evidence within reach and connects the analysis to the tools safety teams already use.
Design principle: Make evidence part of the interaction. Give users a direct path from a finding to its supporting records and back to the analysis.
Ask before changing the scope
A comparison across hospitals can become misleading if one hospital’s data is missing.
Copilot explains the retrieval problem and retries automatically. If the retry fails, it pauses and lets the user decide whether to try again or continue without that hospital.
If the user chooses to continue, the completed analysis identifies the excluded hospital. The change remains visible wherever the result is reviewed.
Design principle: Match autonomy to consequence. Retry a retrieval automatically; ask the user before excluding a facility and changing the meaning of the comparison.
Connect incident patterns with organizational guidance
Incident reports describe events. Internal procedures and reporting guides provide context for understanding patterns across those events.
Copilot finds relevant documents in sources the user can access and compares their guidance with the incident reports. Users can inspect supporting records, read the cited passages, and review the full document.
If a selected source is unsuitable, the user can replace it and rerun the comparison. The earlier result remains available with its original evidence, preserving an audit trail of how the analysis changed.
Design principle: Make agent choices correctable. Users should be able to inspect and replace selected sources, then generate a revised result while preserving the original.
This explores a broader direction: shared organizational context that helps people and AI agents build on the same evidence.
Evaluation framework
I created an evaluation framework around four dimensions: analysis efficiency, source correction, understanding of scope, and recovery. These criteria guided how I designed the Copilot’s interactions and assessed the experience.
| Measure | Evaluation approach |
|---|---|
| Time to a supported finding | Compare the time needed to identify a finding and locate its supporting evidence with the existing workflow. |
| Source correction success | Observe whether users can recognize an unsuitable reference document, replace it, and rerun the comparison. |
| Understanding of scope | Ask users to identify the timeframe, included facilities, and exclusions behind a result. |
| Recovery success | Observe whether users can make an informed choice after a retrieval failure and continue the analysis. |
Moving the work forward, keeping people in control
This work connected an achievable first release to an ambitious product vision. I raised the bar for the experience by showing how AI could help safety teams explore patterns, review evidence, and decide where to investigate further.
My contribution extended from identifying where AI could help to defining how an agent should behave: when to act, what to explain, how to preserve evidence, and when to return a decision to the user.
The result combines personalized assistance with visible progress, inspectable evidence, and human-in-the-loop recovery. Safety teams can move their analysis forward while staying in control of the work.