Sunny
A young Taiwanese woman and Sunny, a glowing golden companion, in a warm Taipei apartment at sunset

MI-087 · AssemblyAI Voice Agent Hackathon

A conversation that gives you the floor.

AssemblyAI understands when you speak. Sunny handles turns and interruptions. Its separate offline rhythm prototype updates a bounded next-response pause policy.

Watch the film View source
Human E2E verified True barge-in verified

The distinction

Speech intelligence, then adaptive conversation.

AssemblyAI supplies real-time speech and turn events. Sunny uses those events to handle responses and yield playback. The numerical learning proof is an offline component, separate from the human-tested voice path.

Submission film

Meet Sunny

95 seconds · Product visualization + sanitized human event proof

01:05–01:20: Real microphone test — sanitized event proof with recorded relative timestamps. No raw tester audio or transcript; the animated woman is not the human tester. Other scenes use synthesized dialogue. Offline pause adaptation is tested separately, not verified in the live human path.

How Sunny works

Speech becomes presence.

AssemblyAI produces speech and turn signals. Sunny handles the accepted turn and yields playback on new speech.

  1. 01Your voice

    Natural speech enters the live audio stream.

  2. 02AssemblyAI

    Real-time SpeechStarted and turn events arrive as the conversation unfolds.

  3. 03Sunny Adaptive Layer

    Accepted turns feed Sunny; SpeechStarted during playback triggers the interruption handler.

  4. 04Playback

    The accepted new turn produces a response after playback yields.

Offline state-update proof

The space between words matters.

The existing offline rhythm prototype updates a rolling pause average and the next-response wait policy. Synthetic 900 ms observations change that policy from 300 to 360 to 414 ms. These are computed policy values, not measured latency; live learning is not verified.

Numerical observationsRolling meanBounded updateExecutable tests

True barge-in

When you speak, Sunny yields.

The interruption path is explicit and easy to understand—even with the sound off.

Visualized verified flow
01

SpeechStarted

AssemblyAI detects the new human speech event.

02

Playback stop / duck

Sunny’s policy yields the audio channel. No latency value is claimed.

03

New turn

The interrupted speech becomes the priority turn.

04

Sunny response

The conversation continues from the new intent.

Architecture

Clear roles. One conversation flow.

AssemblyAI supplies real-time speech intelligence. Sunny handles responses and playback control. Offline rhythm adaptation is a separate evidence track.

AssemblyAI live speech events feed Sunny response and playback control; offline pause adaptation is separately tested

Verified with a human in the loop

Two flows. Both completed.

✓

Human E2E verified

Physical human microphone → AssemblyAI v3 speech events → Sunny → speaker

✓

True human barge-in verified

Playback active → SpeechStarted → stop / duck → new turn → response

Verified functionalHuman microphone E2E · True human barge-in
Verified syntheticExisting offline pause-policy source replay + synthetic contract fixtures
Not yet benchmarkedStop latency · Baseline comparison · Pause-performance metrics

Human evidence is recorded functional validation. Offline learning is separately reproducible from existing source; live automatic learning is not verified. Event timestamps are not acoustic latency measurements. Identity, personality and emotion inference are out of scope for this proof.

Redacted evidence ↗Claim-by-claim status ↗Executable learning proof ↗For Judges — verify Sunny in 60 seconds ↗

AssemblyAI understands when you speak.

Sunny handles your turn; its offline prototype adapts the next pause.

Built with AssemblyAI