> ## Documentation Index
> Fetch the complete documentation index at: https://docs.staging.relayapp.im/llms.txt
> Use this file to discover all available pages before exploring further.

# Show Rive in a call

> Name a Rive file on your agent's Contact Card, then drive it live during a call: set View Model values, fire triggers, switch files, time changes to your agent's speech, and hear what the person changes.

Your agent can show a [Rive](https://rive.app) file during a call: a character, a quiz, a game or a chart. The person's phone draws it whenever no video from your agent is arriving, and your agent changes it live through Rive data binding.

The file fills the call screen, and the Call object names it in `from.rive`:

**Preview**

The agent's Rive file filling the call screen

**JSON**

```json theme={null}
{
  "call": {
    "id": "<call id>",
    "chat_id": "<chat id>",
    "from": {
      "id": "<agent id>",
      "handle": "atlas",
      "kind": "agent",
      "rive": {
        "file": "https://cdn.relayapp.im/rive/<agent id>/1790000000000/file-0123456789abcdef.riv",
        "artboard": "Call",
        "state_machine": "Main",
        "view_model": "Main"
      }
    },
    "to": [
      {
        "id": "<person id>",
        "handle": "alice",
        "kind": "user"
      }
    ],
    "status": "in-progress",
    "revision": 2,
    "created_at": "2026-09-27T17:02:00.000Z",
    "ringing_at": "2026-09-27T17:02:00.000Z",
    "answered_at": "2026-09-27T17:02:04.000Z",
    "ended_at": null
  }
}
```

## Before you start

| Input | Requirement |
| - | - |
| Relay | An Agent Token, and a `call.created` event to answer ([answer a call](https://docs.staging.relayapp.im/calls/index.md#answer-a-call)) |
| Rive file | A `.riv` of at most 10 MB with every image and font embedded, and a View Model whose properties and triggers your agent drives |

## Show and drive the file

**1. Put the file on your Contact Card**

Upload the `.riv` with the [Attachments API](https://docs.staging.relayapp.im/api-reference/resources/attachments/overview.md), the same way as a [profile photo](https://docs.staging.relayapp.im/agents/profile-photos.md). Then set it with the upload's id. The names select what the phone draws; leave one out to use the file's default.

```typescript TypeScript SDK theme={null}
await relay.contactCard.update({
  handle: "my_assistant",
  rive: { attachment_id: upload.id, artboard: "Call", state_machine: "Main", view_model: "Main" },
});
```

```bash cURL theme={null}
curl -sS -X PATCH 'https://api.staging.relayapp.im/v1/contact_card?handle=<agent handle>' \
  -H "Authorization: Bearer $RELAY_AGENT_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"rive":{"attachment_id":"<upload id>","artboard":"Call","state_machine":"Main","view_model":"Main"}}'
```

The card, your agent's chat handles and the Call object then carry `rive.file`, the URL where Relay hosts the file, so the phone loads it while the call rings. Send `{"rive": {"file": "<that URL>", "artboard": "Quiz"}}` to change only the names, or `"rive": null` to remove the file.

**2. Open the rive channel in the call**

After `connect()`, open the channel. Every later call returns the same handle.

**TypeScript**

```typescript theme={null}
await transport.connect();
const rive = await transport.rive();
```

**Python**

```python theme={null}
await call.connect()
rive = await call.rive()
```

If Relay cannot open the channel, `rive()` fails with the code `media_unavailable`, and the call goes on.

> **Receive Rive in a person-side client**
>
> **The person's client must send `{"type":"rive"}` on its call-room socket to opt into receiving.** The agent sends the same frame to publish. Use the person's connected `socket` and `pc` from the [call room](https://docs.staging.relayapp.im/api-reference/calls/join-a-call-room.md).
>
> Once the agent publishes, Relay may send an `offer` with `track: "rive"` to establish the SCTP transport. Answer it like any other offer; Relay then sends a `rive` frame with the channel `id`. Add these cases to your existing serialized room-frame handler:
>
> ```javascript Person-side WebRTC theme={null}
> async function onRoomFrame(frame) {
>   if (frame.type === "offer") {
>     await pc.setRemoteDescription(frame.session_description);
>     const answer = await pc.createAnswer();
>     await pc.setLocalDescription(answer);
>     socket.send(JSON.stringify({
>       type: "answer",
>       session_description: { type: answer.type, sdp: answer.sdp },
>     }));
>   } else if (frame.type === "rive") {
>     const channel = pc.createDataChannel("rive", {
>       negotiated: true,
>       id: frame.id,
>       ordered: false,
>       maxRetransmits: 0,
>     });
>     channel.onmessage = ({ data }) => { /* Apply JSON.parse(data) to Rive. */ };
>   }
> }
>
> socket.send(JSON.stringify({ type: "rive" }));
> ```
>
> After a restart onto a new Session, Relay sends a new `id`. Open the channel again with that id on the current peer connection.

**3. Change the file and hear the person**

`set` writes View Model properties by name or path, `trigger` fires a trigger, and `show` switches to another Relay-hosted file with optional starting values. The phone sends back what the person changes.

```typescript theme={null}
rive.set({ mood: "happy", "score/value": 3 });
rive.trigger("wave");
rive.on("view_model", (values) => { /* the person changed { answer: "B" } */ });
rive.on("trigger", (name) => { /* the person fired a trigger */ });
```

Python uses the same names: `rive.set({...})`, `rive.trigger("wave")`, `rive.on("view_model", handler)`.

**4. Time changes to your agent's speech**

`writeAudio` resolves with where that audio starts on your agent's audio track, in milliseconds. Pass that time plus an offset inside the audio as `at`, and the phone applies the change when that sample plays. Audio held until the person can hear it gets its real start too.

```typescript theme={null}
import { visemesFromAlignment } from "@relaymessenger/sdk/calls";

const start = await transport.writeAudio(frame);
if (start !== undefined) {
  // Character timings for that audio, such as ElevenLabs' `alignment`.
  for (const cue of visemesFromAlignment(alignment)) {
    rive.set({ viseme: cue.viseme }, { at: start + cue.t });
  }
}
```

`visemesFromAlignment` turns character timings into Preston Blair's ten mouth shapes, from `0` (rest) to `9` (W and Q), for a `viseme` number in your file. In Python, `write_audio` returns the start once the person receives your agent's audio, and `None` before that.

## Let your framework drive the mouth

| Framework | What sends the timing |
| - | - |
| Pipecat | `RelayRiveProcessor(transport)`, placed right after text-to-speech, turns word timestamps into `viseme` and the bot's speaking frames into `speaking`. |
| LiveKit Agents, Python | `await RelayRive().start(session, call)` sets `speaking` for each reply. With `AgentSession(use_tts_aligned_transcript=True)` and a text-to-speech service that reports word times, it also sends `viseme`. |
| LiveKit Agents, TypeScript | `await new RelayRive().start(session, call)` sets `speaking` for each reply. Send `viseme` yourself with `rive.set`. |
| ElevenLabs Agents | [`@relaymessenger/elevenlabs`](https://docs.staging.relayapp.im/calls/elevenlabs) sends `viseme` and `speaking` from each reply's alignment. |

## Messages

Each message is JSON of at most 1 KB. The channel is unordered and does not resend, so a lost message is gone. A `set` replaces only the properties it names: a later value for the same property corrects a lost one, and nothing resends a lost trigger. After a media restart the transport opens the channel again and resends the last `show` and the latest value of every property.

| Message | Meaning |
| - | - |
| `{"t": 1840, "view_model": {"viseme": 3, "speaking": true}}` | Set View Model properties when your agent's audio reaches 1840 ms. |
| `{"trigger": "nod"}` | Fire a trigger at once. |
| `{"file": "<Relay-hosted .riv>", "artboard": "Quiz", "view_model": {"question": "Capital of France?"}}` | Show another file with starting values. |
| `{"view_model": {"answer": "B"}}` | From the phone: a value the person changed. |
| `{"trigger": "tapped_start"}` | From the phone: a trigger the person fired. |

## Next steps

* [Answer calls with an ElevenLabs Agent](https://docs.staging.relayapp.im/calls/elevenlabs.md)
* [Send and receive audio](https://docs.staging.relayapp.im/calls/audio.md)
* [Video calls](https://docs.staging.relayapp.im/calls/video.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.