From cc7c0b91f7f81fa4072c604e957a5a70d63475f6 Mon Sep 17 00:00:00 2001 From: juancarmore Date: Tue, 27 Oct 2026 18:09:41 +0100 Subject: [PATCH] openvidu-dashboard: the writer stores values starting with $ as received --- .../store-values-as-received/.openspec.yaml | 2 + .../store-values-as-received/design.md | 50 +++++++++++++++++++ .../store-values-as-received/proposal.md | 38 ++++++++++++++ .../specs/server-analytics-writer/spec.md | 10 ++++ .../changes/store-values-as-received/tasks.md | 19 +++++++ 5 files changed, 119 insertions(+) create mode 100644 openvidu-dashboard/openspec/changes/store-values-as-received/.openspec.yaml create mode 100644 openvidu-dashboard/openspec/changes/store-values-as-received/design.md create mode 100644 openvidu-dashboard/openspec/changes/store-values-as-received/proposal.md create mode 100644 openvidu-dashboard/openspec/changes/store-values-as-received/specs/server-analytics-writer/spec.md create mode 100644 openvidu-dashboard/openspec/changes/store-values-as-received/tasks.md diff --git a/openvidu-dashboard/openspec/changes/store-values-as-received/.openspec.yaml b/openvidu-dashboard/openspec/changes/store-values-as-received/.openspec.yaml new file mode 100644 index 000000000..97e364202 --- /dev/null +++ b/openvidu-dashboard/openspec/changes/store-values-as-received/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-27 diff --git a/openvidu-dashboard/openspec/changes/store-values-as-received/design.md b/openvidu-dashboard/openspec/changes/store-values-as-received/design.md new file mode 100644 index 000000000..0e2178a2d --- /dev/null +++ b/openvidu-dashboard/openspec/changes/store-values-as-received/design.md @@ -0,0 +1,50 @@ +# Design + +## Context + +- Every entity update of the writer is a pipeline of `$set` stages built by `stage()` from the assignments of a + `fields` set. Values are mixed: expressions from the helpers of `expr.go` (`$ifNull`, `$cond`, `$setUnion`…), field + paths and variables (`$name`, `$$this.v`, `$$REMOVE`) and data from the event (names, identities, errors, metadata, + encoded messages, attributes). +- Nothing tells data apart from expressions, so MongoDB evaluates a data string starting with `$` and a data document + whose keys look like operators or paths. + +## Goals / Non-Goals + +**Goals:** +- Data is never evaluated, whatever it holds, without touching each of the call sites. + +**Non-Goals:** +- Changing how expressions are built, or the classic updates of the rollups and the heartbeat, where values are never + evaluated. + +## Decisions + +### Typed references, escaped stages + +Field paths and variables get their own type, `ref`, returned by `field()` and used for `$$REMOVE` and the `$$this` +variables. `stage()` walks its assignments and wraps in `$literal` every plain string that starts with `$`; a `ref` +becomes its string. The walk enters documents and arrays, and leaves an existing `$literal` as it is. + +Alternative rejected: wrapping data at each call site. There are about 300 assignments, and a missed one fails only +with an unusual value. + +### Maps wrapped at the source + +Participant attributes are the only document whose keys come from the client. The projection wraps them in +`$literal`, so their keys are neither operators nor paths. The other documents (encoded messages, the request summary, +outputs, permissions, client info) have fixed keys and are walked like any expression. + +### Data model + +- No collection, field, index, retention or schema-version change. Values that were lost or evaluated are now stored + as received. +- Migration from legacy events: the legacy migration projects with the same code, so migrated documents get the fix + too. +- Performance: one walk of each stage before it is sent, linear in its size; negligible next to the write. + +## Risks / Trade-offs + +- [A future expression writes a field path as a plain string] → It would be stored as a string instead of evaluated. + The integration suites compare every projected document with its expected contract fields, so the change shows up + there. diff --git a/openvidu-dashboard/openspec/changes/store-values-as-received/proposal.md b/openvidu-dashboard/openspec/changes/store-values-as-received/proposal.md new file mode 100644 index 000000000..056dc604f --- /dev/null +++ b/openvidu-dashboard/openspec/changes/store-values-as-received/proposal.md @@ -0,0 +1,38 @@ +# Proposal + +## Why + +The writer updates entity documents with aggregation pipelines, where a value is an expression. A string that starts +with `$` is read as a field path or a variable, and the keys of a map as operators or paths. A room named `$demo` +loses its name, an identity `$$ROOT` is replaced by the document being updated, and a participant attribute whose key +starts with `$` makes the update fail. + +## What Changes + +- Every value the projectors write is stored as received, after the allow-list and its redactions: strings starting + with `$`, and map keys that start with `$` or hold a dot, included. + +Scope excludes the following: +- Raw events, which are inserted, not updated, and keep their values already. +- Field names the writer derives from fixed vocabularies (kinds, sources, families, API methods). + +Edition: CE and PRO (writer). Repositories: openvidu-livekit (`openvidu/analytics/projection/`) and the specs of the +dashboard. + +CHANGELOG: none. The analytics data model has not shipped in a release. + +## Capabilities + +### New Capabilities + +None. + +### Modified Capabilities + +- `server-analytics-writer`: values are stored as received. + +## Impact + +- Writer: the expression helpers and the stage builder of `openvidu/analytics/projection/`, and the participant + attributes. Integration tests of the writer. +- Dashboard: none. diff --git a/openvidu-dashboard/openspec/changes/store-values-as-received/specs/server-analytics-writer/spec.md b/openvidu-dashboard/openspec/changes/store-values-as-received/specs/server-analytics-writer/spec.md new file mode 100644 index 000000000..3af116d8a --- /dev/null +++ b/openvidu-dashboard/openspec/changes/store-values-as-received/specs/server-analytics-writer/spec.md @@ -0,0 +1,10 @@ +## ADDED Requirements + +### Requirement: Values stored as received +Edition: CE and PRO. Every value the projectors write into a document SHALL be stored as received, after the +allow-list and its redactions, including strings that start with `$` and map keys that start with `$` or hold a dot. + +#### Scenario: Values starting with a dollar sign +- **GIVEN** `store_metadata: true`, a room `$room` with metadata `$$ROOT`, a participant `$$ROOT` named `$name` with metadata `$metadata` and attributes `{ "$role": "$host", "team.name": "$blue" }`, and an egress of the room that fails with error `$error` +- **WHEN** their events are projected +- **THEN** the room has `name: "$room"`, `config.metadata: "$$ROOT"` and one unique identity, the participant has `identity: "$$ROOT"`, `name: "$name"`, `metadata: "$metadata"` and those attributes, and the egress has `error: "$error"` diff --git a/openvidu-dashboard/openspec/changes/store-values-as-received/tasks.md b/openvidu-dashboard/openspec/changes/store-values-as-received/tasks.md new file mode 100644 index 000000000..f4700d39c --- /dev/null +++ b/openvidu-dashboard/openspec/changes/store-values-as-received/tasks.md @@ -0,0 +1,19 @@ +# Tasks + +## 1. Writer (openvidu-livekit) + +- [x] 1.1 Add the `ref` type to `openvidu/analytics/projection/expr.go` for `field()`, `$$REMOVE` and the `$$this` + variables of the projection, and escape every other string starting with `$` in `stage()`. Wrap the participant + attributes in `$literal`. Verify with the unit tests of `openvidu/analytics/...` and the lint of the fork. +- [x] 1.2 Add a writer integration test with a room, a participant and an egress whose values start with `$`. Verify + with the analytics integration tests, the migration and contract suites included. + +## 2. Docs and CHANGELOG + +- [x] 2.1 No CHANGELOG line: the analytics data model has not shipped. No `AGENTS.md` change. Verify by reading the + change against both files. + +## 3. Integration check + +- [x] 3.1 Run the spec validation and the format check of the dashboard, and `typings/contract/writer-check.sh` against + the fork. Verify that they pass.