Easing nervous first-⁠timers into a live argument with an AI coworker

Cornell Future of Learning Lab, 2026

When your coworker jumps in mid-thought. Participants practice handling conflict at work with AI coworkers who talk back. I redesigned the onboarding, built the researchers’ replay view and measured the coworkers in live voice tests.

162 issues fixed before data collection

One was that session recordings could be opened without a login.

Role
Undergraduate researcher and full-⁠stack developer, AI-⁠Ready Workforce Initiative
Team
The lab’s principal investigator and research staff, with Anthropic Research
Timeline
August 2026 to now
What changed
First-timers used to stall before the call. Now they get through.

In short

  • The study runs with Anthropic Research. I brought in five new company partners.
  • The coworkers speak through Gemini Live but had only been measured in text. Out loud, they jumped in mid-thought.
The Evidence Trace with a made-up session, open on the first conversation, with Riley. Each turn is a row with its time in seconds, the participant’s words, the AI director’s note to the actor and Riley’s reply. The note at 58.9 seconds tells Riley to push for a public reply-all and offer to draft it, and it’s marked steered. Two later turns are marked unsteered.

The numbers

12participants in usability testsThink-aloud sessions.
22 percent to 0 percentfalse audio cut-offs in the live testsBefore and after room-adaptive voice detection. Real interruptions that got through rose from 25 to 56 percent.

My part

  1. Redesignedthe onboarding, from consent to the live call, and changed it wherever a first-timer got stuck.12 participants
  2. Builtthe Evidence Trace, which shows each turn of a session as one row.Researcher dashboard
  3. Measuredthe coworkers live and fixed the turn-taking and audio problems that came up.False cut-offs 22% to 0%

Sam couldn’t tell your voice from keyboard noise.

Cut off with nobody speaking
22% → 0%
Real interruptions that got through
25% → 56%

Sam is talking, and a keyboard is clicking beside the participant’s microphone. In the first version, the voice detector had a fixed threshold and took the clicks for the participant. Sam stopped mid-sentence with nobody speaking, and the rest of the reply was never heard. On Sam’s next line, the participant did cut in, with “Sorry, can I stop you there?” The detector was still busy with the typing and missed it, so Sam talked over them. With room-adaptive voice detection, the threshold is set from the room’s own noise. Sam finished the reply and stopped as soon as the participant spoke.

Sample lines from the study’s Taken credit scenario. The numbers come from the live tests. Turns cut off with nobody speaking went from 7 of 32 to 0 of 35. Real interruptions that got through went from 5 of 20 to 10 of 18.

How long should Sam wait before answering?

Sam waits for
  • 0.9 s. Sam answered before you’d finished your thought, then got cut off when you kept going. With the first settings, every hesitant line in the live tests split this way: 18 of 18.
  • 1.5 s. Sam waited through both pauses and answered 1.5 seconds after you finished. That’s longer than every pause the live tests measured (0.7 to 1.3 seconds), and it’s the longest wait that kept a reply from feeling slow. It’s the setting on Gemini Live now.
  • 2.5 s. Sam waited through both pauses too, but then left 2.5 seconds of silence before every answer.

A participant who hesitates says, “So, the deck this morning… the analysis was mine… and I’d like my name on it,” with a 0.8-second pause and then a 1.2-second pause. Sam answers once the silence lasts as long as the wait you pick.

Sample lines from the study’s Taken credit scenario. The pauses are as long as the ones measured in the live tests.

Clips

The onboarding in the lab’s build: the quiet-room notice, the audio and camera check, then the scene card. Recorded from the lab’s code on 27 October 2026 with made-up test data. A drawing stands in for the camera.
A group call in the lab’s build. The cue at the top says whose turn it is, and each coworker’s words come up as captions. Recorded from the lab’s code on 27 October 2026 with scripted lines and a drawing for the camera.
The Evidence Trace in the lab’s build, one row per turn, with each of the director’s notes marked steered or unsteered. Recorded from the lab’s code on 27 October 2026 with a made-up session.

The problem

The platform has two kinds of users, and they never meet.

One is a nervous participant in a headset who’s about to argue with an AI coworker named Sam. Each coworker has two parts: an actor the participant hears, and a director that tells the actor how to play each turn.

The other is the researcher. They need to know what happened in every turn, down to whether a director’s note reached the actor late.

The first version failed both of them. Participants stalled between consent and the call, and researchers couldn’t line up the audio with the logs.

Decision log

Chosen

to measure the coworkers live, with a scripted participant who hesitates and gets lost.

Rejected

text-only tests.

Why

the coworker jumped in during mid-thought pauses. Room noise, like a keyboard by the microphone, also cut off its own replies. The text tests hadn’t shown either problem.

Result

the coworker now waits through a pause. With room-adaptive voice detection, false cut-offs fell from 22 percent to zero.

Chosen

to pick the voice model by talking to the candidates.

Rejected

their spec sheets and published benchmarks.

Why

what matters is whether a coworker sounds like it’s listening and whether the director’s notes reach it. No benchmark measures either.

Result

the live trials found that the director’s notes never reached the Gemini coworkers. The same setup also caught bugs that were dropping participant audio.

Chosen

the Evidence Trace. It gives each turn one row, with the participant’s words, the director’s note and the actor’s reply on one timeline. It also has fidelity checks, and flags for notes that arrived late or leaked into the conversation.

Rejected

a transcript plus a separate log download.

Why

if a researcher scoring a session can’t see that a note arrived late, they’ll read the delay as the character’s personality. That error would carry into the study’s conclusions.

Result

researchers can replay any session against its logs in one view. The build marks each note as steered or unsteered. The Late flag needs the platform to time its notes, which it doesn’t do yet. The leak flag isn’t built.

My design for the Evidence Trace, with sample sessions. Down the left are the sessions, each with its status (complete, partial, flagged or failed). In the middle, one conversation with Sam is a table with one row per turn: its time in seconds, the participant's words, the director's note to the actor with its state code, how many milliseconds the note took to arrive, a badge reading Applied (or Late, on the turn at 58.9 seconds), and the actor's reply. On the right are the fidelity checklist and the session's health. A replay timeline runs along the bottom beside the video.
My design for the Evidence Trace, 25 August 2026, with sample sessions. A note marked Late reached the actor after it had started its reply, so that turn wasn’t steered.
Chosen

to strip the call screen down to what a participant needs: who they’re talking to, that it’s recording, and a way to leave.

Rejected

the usual video-call controls.

Why

a self-view, a mute button and a timer all gave nervous participants something to fiddle with instead of talking.

Result

first-time participants now get through the flow. The revisions came out of the 12 think-aloud sessions. The build kept a self-view and a running timer on the call.

First, a device check: the camera, microphone and speakers, each with a green check mark, then a Start button above a note that recording is shown at all timesThen a scene card before each conversation: “Later that afternoon. You run into Sam by the elevators. Sam presented your analysis again this morning.” and a Continue buttonThe call: Sam's tile with his initial, a Listening meter under it, the REC (recording) light, “Conversation 2 of 2”, an Audio issue link and a Leave conversation button. Nothing else is on screen: no self-view, mute button or timerLeaving: a dialog over the call asks “Leave the conversation with Sam? You won't be able to rejoin this conversation.” with Stay and Leave buttons
My design for the onboarding and the call, 25 August 2026: a device check, a scene card before each conversation, and a call screen with only who you’re talking to, that it’s recording, and a way to leave.
First: a notice that you need a quiet room because the microphone hears everyone. It asks you to keep the conversation fictional, with no real names, real events or contact details, and to steer back to the scenario if it drifts. It ends by saying each conversation lasts 7 to 12 minutes and that you can leave the study at any time, above the button to go onSecond: the audio and camera check with all three checks passed. The microphone reads We can hear you, the speakers Sound works and the camera Camera on, with a drawing of a person in headphones in its preview. Continue is unlocked, with a Skip the check link beside itThird: the card about the Taken credit scenario. Sam has twice presented your analysis as his own, and your goal is to get credit from now on and keep working well with him. You speak with Riley for up to 1 minute, then with Sam in the hallway for up to 11 minutes, and you start each conversation. A reminder that it's fictional sits above the I am ready button
The onboarding in the lab’s build after my redesign, rendered from the lab’s code on 27 October 2026 with made-up test data. A drawing stands in for the participant’s camera.

Not in the first version

These were designed but left for later: a live view of several sessions at once, analytics across sessions, a prompt editor, and rater assignment and export.

What I would do next

I’d run the researcher side through the same 12-⁠person usability protocol. So far, the dashboard’s only tester is me.

My raters' screen during stage 1 of 4, marking a clip. The video is labeled Character 2 speaking. On the bar under it, a dashed box runs from where the clip started to the playhead. Below that are Pause, the time (3:04 of 5:14), Cancel and End clip, and a note: Clip started at 2:47. Press End clip when the moment is overThe same screen with a line added as an example. The bar under the video has three examples marked in blue. Below it are Play, the time (2:21 of 5:14) and a Mark a clip button. The transcript below reads Adding examples to question 1. The participant's line at 2:19 is highlighted and marked Added, and the participant's next line has an Add buttonAfter a rater submits stage 1. The menu shows stage 1 Submitted, stage 2 Not started, and stages 3 and 4 locked until the one before is done. The page says Stage 2 is open, with a Start stage 2 buttonOn a phone, the first step of the tour: a card at the bottom reads Watch the conversation and says to press Play or click the video to start, with Skip tour, 1 of 5 and Next. Behind it, dimmed, the transcript opens with the participant saying Hey, Riley. What's up?
My raters’ screen as of 27 October 2026. The first time a rater opens it, a short tour shows them around in five steps. The four stages open one at a time. Any score above 1 needs at least one example, either a line the participant said or a clip of them speaking. Each stage ends with a review before the rater submits it. I left the questions out of these pictures because the lab hasn’t cleared them for public use. The conversation is made up, and the screen isn’t connected to study data yet.