2026-10-25 16:35:47 +01:00
# Data generator
Fills a MongoDB database with analytics data that follows the data model contract of `@openvidu-dashboard/typings` : a
deterministic scenario S, M or L over a number of days ending now. The backend integration tests use it for their query
plan and latency checks, and `./dashboard.sh demo` uses it for the local demo database.
Before inserting, `src/schema.ts` creates every collection, index and TTL of the contract and gives the plain
collections the contract validator with `validationAction: error` , so a document that drifts from the contract fails
the run (`track_stats` takes no validator as a time-series collection). `src/contract.ts` turns each model document
into its stored shape: the model's bookkeeping removed, absent fields omitted and every number in the BSON type its
schema declares.
The package is also a library for the backend integration tests (`dashboard-ce/backend/test/`): `./generator`
(`generate(db, options)`), `./context` (`queryContext(db)`: the data range and sample identifiers), `./plan` (explain
summaries used by the `expectIndexBacked` harness) and `./schema` (`createCollections()`).
Requirements: the workspace installed (`./dashboard.sh install`), a reachable MongoDB (8.x; time-series
collections are used when the server accepts them) and a user allowed to create and drop the target
database. Run the commands from this directory.
## Usage
```sh
pnpm generate --uri "$MONGO_URI" --db ov_data_s --scenario S --days 30 --drop
pnpm generate --uri "$MONGO_URI" --db ov_data_m --scenario M --days 7 --drop
pnpm generate --uri "$MONGO_URI" --db ov_data_l --scenario L --days 1 --drop --stats-mode track
```
2026-11-02 20:39:42 +01:00
| Option | Default | Meaning |
| ----------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `--uri` , `--db` | required | Connection string and target database |
| `--scenario S\|M\|L` | required | Rooms per day, sessions per day and average room size (table below) |
| `--days <n>` | required | Days of history ending now |
| `--stats-mode participant\|track` | `participant` | One `track_stats` document per participant and direction, or per track |
| `--seed <n>` | `1` | Seed of the pseudo-random generator (mulberry32); same seed, same data |
| `--retention-events-days <n>` | `7` | Only raw events inside this window are written (TTL index on `ts` ) |
| `--retention-stats-days <n>` | `7` | Only stats inside this window are written (`expireAfterSeconds`) |
| `--retention-tracks-days <n>` | `7` | `expiresAt` of tracks and subscriptions; expired ones are not written |
| `--retention-entities-days <n>` | `90` | `expiresAt = closedAt + retention` on closed entities |
| `--retention-rollups-5min-days <n>` | `7` | Only 5-minute rollups inside this window are written (TTL index on `bucket` ) |
| `--drop` | off | Drop the database first |
| `--plain-stats` | off | Force the plain-collection fallback for `track_stats` |
| `--five-minute-rollups` | off | Write `analytics_5min` inside the 5-minute rollup window |
openvidu-dashboard: media node labels, and nothing about nodes in CE
The contract gains mediaNode on nodes, rooms and participants, stoppedAt on nodes and the
partial mediaNode index that replaces the nodeId one; the scenario's two servers are
labelled. The status reports the writer without node counts and with stoppedAt when the
server stopped, the rooms list filters by mediaNode, list items carry it, the overview
groups by it for editions and a node scope's media statistics coverage follows it.
CE renders nothing about nodes without per-node-breakdown: no nodes line, By node card,
Node select, scope selector or Node columns, not even locked; the Overview reads "Server
stopped at" for a stopped server. The generator gains --media-nodes, which node rollups
now require.
2026-11-04 12:13:58 +01:00
| `--node-rollups` | off | With `--media-nodes` : write the `node:<label>` scope beside `global` in every rollup |
2026-11-02 20:39:42 +01:00
| `--room-sums` | off | Write `rooms.participantsSum` and `rooms.durationMsSum` at room end |
| `--active-window <minutes>` | off | Rooms created in the last minutes stay active (two of three); without it, the most recent 10 % of the rooms are active |
| `--min-active-rooms <n>` | 0 | With `--active-window` : rooms added to the window when the hour of day leaves fewer than `n` active rooms |
| `--nodes-without-media-stats <n>` | 0 | The last `n` of the three nodes report no media statistics: `mediaStats: false` , no media `track_stats` , bytes or quality |
openvidu-dashboard: media node labels, and nothing about nodes in CE
The contract gains mediaNode on nodes, rooms and participants, stoppedAt on nodes and the
partial mediaNode index that replaces the nodeId one; the scenario's two servers are
labelled. The status reports the writer without node counts and with stoppedAt when the
server stopped, the rooms list filters by mediaNode, list items carry it, the overview
groups by it for editions and a node scope's media statistics coverage follows it.
CE renders nothing about nodes without per-node-breakdown: no nodes line, By node card,
Node select, scope selector or Node columns, not even locked; the Overview reads "Server
stopped at" for a stopped server. The generator gains --media-nodes, which node rollups
now require.
2026-11-04 12:13:58 +01:00
| `--media-nodes` | off | Label the nodes `media-node-10-0-0-1` to `-3` and add an earlier stopped process of the first one (a restart) |
2026-10-25 16:35:47 +01:00
| Scenario | Rooms per day | Sessions per day | Average room size |
| -------- | ------------- | ---------------- | ----------------- |
| S | 200 | 1 000 | 4 |
| M | 2 000 | 20 000 | 6 |
| L | 5 000 | 50 000 | 8 |
What it writes (`analytics_5min` stays empty without `--five-minute-rollups` ):
- `rooms` : created over the days with a diurnal curve (office-hours peak); the most recent 10 % are
`active` , the rest `closed` with `endReason` and `closedBy` . Id arrays and counters (`participantSids`,
`activeParticipantSids` , `identities` , `participants.*` , `trackSids` , `egressIds` , `ingressSessionIds` ,
`features` ) are derived from the generated entities; participant counters count STANDARD sessions and `byKind`
every kind, as the writer does (also in the rollups).
- `participants` : unique identities per room follow a Poisson distribution around the scenario average;
the difference between sessions/day and rooms/day ×
(`DUPLICATE_IDENTITY` leave plus a new session of the same identity). Sessions last ~20 min, 3 % are
aborted, 95 % are `STANDARD` ; rooms with egress/ingress/agent/SIP get the matching participant kinds.
Client (sdk/os/browser/device/network), connection type, connect time, disconnect reason and resumes
follow fixed distributions.
- `tracks` (CAMERA video + MICROPHONE audio per publishing participant, with simulcast layers and final
publish stats) and `track_subscriptions` (every participant subscribes to the tracks of the other
identities present at the same time; 1 % failed).
- `egresses` (10 % of rooms; mostly `room_composite` ; `complete` /`failed`/`aborted`, `active` in active
rooms), `ingresses` and `ingress_sessions` (2 % of rooms).
- `events` : lifecycle events of rooms, participants, tracks, subscriptions, egresses and ingresses with
small payloads, only inside the events retention window.
- `track_stats` : one document per 30 s per participant and direction (`participant` mode) or per track and
direction (`track` mode), only inside the stats retention window, with plausible score/bytes/rtt/jitter.
Time-series collection (`timeField: ts`, `metaField: meta` , `granularity: seconds` ) with
`expireAfterSeconds` ; plain collection with a TTL index on `ts` when the server rejects time-series.
openvidu-dashboard: media node labels, and nothing about nodes in CE
The contract gains mediaNode on nodes, rooms and participants, stoppedAt on nodes and the
partial mediaNode index that replaces the nodeId one; the scenario's two servers are
labelled. The status reports the writer without node counts and with stoppedAt when the
server stopped, the rooms list filters by mediaNode, list items carry it, the overview
groups by it for editions and a node scope's media statistics coverage follows it.
CE renders nothing about nodes without per-node-breakdown: no nodes line, By node card,
Node select, scope selector or Node columns, not even locked; the Overview reads "Server
stopped at" for a stopped server. The generator gains --media-nodes, which node rollups
now require.
2026-11-04 12:13:58 +01:00
- `analytics_hourly` and `analytics_daily` (global scope, plus one scope per media node label with
`--media-nodes --node-rollups` ) aggregated from the generated entities; bytes and quality are spread over the
hours each session was connected. `--room-sums` adds the room size and duration sums; `--five-minute-rollups`
adds `analytics_5min` (lifecycle counters, the `participants.connected` gauge, bytes and quality per five minutes)
inside the 5-minute rollup window. `analytics_daily` gets no TTL index: daily rollups are kept forever.
`generate()` takes the same switches as `rollups: { fiveMinute, nodes, roomSums }` and `mediaNodes` .
- `nodes` (3 nodes, plus the stopped earlier process of the first one with `--media-nodes` ) and `schema_meta`
(`schema`, `retention` , `capabilities` , `config` ).
2026-10-25 16:35:47 +01:00
Documents are inserted in batches of 5 000 with `insertMany(ordered: false)` . The output shows progress
and a final summary with documents per collection, elapsed time and inserts per second. The generator
keeps only one room in memory at a time, so scenario L fits in a few hundred MB of heap.
## Files
| File | Role |
| ------------------ | -------------------------------------------------------------------------------------------- |
| `src/schema.ts` | Collections, indexes and validators of the contract, `createCollections()` with the fallback |
| `src/contract.ts` | Stored shape of a model document: bookkeeping removed, nulls omitted, BSON numeric types |
| `src/model.ts` | Scenarios, document types and the per-room entity generator |
| `src/derive.ts` | Raw events, stats samples and rollups derived from a generated room |
| `src/generator.ts` | `generate(db, options)` and the batched writer |
| `src/generate.ts` | Generator CLI |
| `src/context.ts` | `queryContext(db)` : data range and sample identifiers for query plan tests |
| `src/plan.ts` | Explain summaries: plan stages, indexes, documents examined |
| `src/random.ts` | mulberry32 generator and sampling helpers |
| `src/args.ts` | CLI argument parsing |
