mirror of https://github.com/OpenVidu/openvidu.git
107 lines
6.0 KiB
Markdown
107 lines
6.0 KiB
Markdown
# Changelog
|
|
|
|
Every release groups its `Added`, `Improved` and `Fixed` entries by the cloud they apply to:
|
|
|
|
- **All clouds**: changes that behave the same on every cloud.
|
|
- **AWS**, **Azure**, **Google Cloud**, **Oracle Cloud** and **DigitalOcean**: what is specific to that
|
|
cloud's templates.
|
|
|
|
The AWS, Azure and Google Cloud templates live in this folder. The Oracle Cloud and DigitalOcean templates live
|
|
in the [openvidu-oracle](https://github.com/OpenVidu/openvidu-oracle) and
|
|
[openvidu-digitalocean](https://github.com/OpenVidu/openvidu-digitalocean) repositories.
|
|
|
|
## 3.10.0 (unreleased)
|
|
|
|
### Added
|
|
|
|
#### All clouds
|
|
|
|
- Elastic and High Availability deployments replace an unhealthy Media Node with a new one automatically,
|
|
keeping the same number of Media Nodes. Before, a broken Media Node stayed in the cluster until it was
|
|
replaced by hand. Every Media Node runs a health watchdog (the `openvidu-media-health` systemd service)
|
|
that checks its media server every 30 seconds and asks the cloud to replace the node when:
|
|
- the media server does not accept connections for 5 minutes,
|
|
- the media server answers with errors or timeouts for 10 minutes (not counted while the node's CPU is
|
|
saturated, because an overloaded media server briefly reports itself as not ready),
|
|
- the node's installation failed (after a 10 minute wait, so the failure can be inspected and a persistent
|
|
problem does not create new nodes in a loop),
|
|
- or the node's installation has not finished 90 minutes after it booted.
|
|
- Safeguards of the Media Node watchdog:
|
|
- No Media Node is replaced while no Master Node is reachable: a Master Node outage makes every Media Node
|
|
fail its health check at once, and a new node could not join the cluster either. Once a Master Node is
|
|
back, the media server gets its full time window to reconnect before the watchdog acts.
|
|
- Failures are not counted during the first 15 minutes after OpenVidu (re)starts on the node.
|
|
- The watchdog pauses itself while a node is drained during a normal scale-in.
|
|
- `journalctl -u openvidu-media-health` on a Media Node shows what it is doing, and
|
|
`sudo touch /etc/openvidu/media-health.disabled` pauses it (for example during manual maintenance);
|
|
deleting the file resumes it.
|
|
- Elastic and High Availability deployments on AWS, Azure and Google Cloud can run a fixed number of Media
|
|
Nodes, as DigitalOcean and Oracle Cloud already could, with the same parameter: `FixedNumberOfMediaNodes`
|
|
on AWS, `fixedNumberOfMediaNodes` on the other clouds. With a value greater than 0 the deployment runs
|
|
exactly that many Media Nodes with no autoscaling, and the initial, minimum, maximum and target CPU
|
|
parameters are ignored. The default, 0, keeps autoscaling. An unhealthy Media Node is still replaced.
|
|
Changing the number, or switching between the two modes, is a redeploy of the template with the new value.
|
|
Raising the number adds Media Nodes. On AWS and Google Cloud, lowering it drains the Media Nodes it removes
|
|
gracefully, as in a scale-in: each one stops accepting new Rooms and is removed when its ongoing Rooms,
|
|
recordings and ingests end. Azure does not accept a number below the Media Nodes running, because they
|
|
protect themselves from scale-in: drain the extra ones first (manual scale, then
|
|
`/usr/local/bin/stop_media_node.sh` on each, which removes itself the same graceful way) and then redeploy.
|
|
The administration documentation of each cloud describes the steps.
|
|
|
|
#### AWS
|
|
|
|
- An unhealthy Media Node is marked unhealthy in its Auto Scaling Group, which drains it and launches a new
|
|
one.
|
|
|
|
#### Azure
|
|
|
|
- An unhealthy Media Node is reimaged in its Virtual Machine Scale Set, so the number of instances never
|
|
changes.
|
|
|
|
#### Google Cloud
|
|
|
|
- An unhealthy Media Node is recreated in its managed instance group with a fresh disk. The group's native
|
|
autohealing (TCP health check on port 7880), now also enabled in High Availability deployments, covers a
|
|
Media Node VM that stops responding altogether.
|
|
|
|
#### Oracle Cloud
|
|
|
|
- An unhealthy Media Node is terminated and its instance pool launches a new one. With a fixed number of
|
|
Media Nodes, the node is detached from the pool, which launches its replacement, and then terminated.
|
|
|
|
#### DigitalOcean
|
|
|
|
- With autoscaling, an unhealthy Media Node is tagged so the autoscaler deletes it and creates a new one. With
|
|
a fixed number of Media Nodes, the Droplet is rebuilt in place.
|
|
|
|
### Fixed
|
|
|
|
#### All clouds
|
|
|
|
- High Availability: publishing to an RTMP ingress failed. OpenVidu returns the ingress URL on port 1945
|
|
(`rtmps://<your-domain>:1945/rtmp`), but the load balancer only accepted RTMP on port 1935. The load
|
|
balancer now accepts RTMPS on port 1945. Publish to the URL OpenVidu returns when the ingress is created, and
|
|
allow port 1945 in any firewall in front of your publishers.
|
|
|
|
#### AWS
|
|
|
|
- Single node and Elastic: the Elastic IP was attached only after the instance had started booting, so the
|
|
domain could be derived from a temporary public IP, or the instance could start without Internet access and
|
|
fail its installation. The Elastic IP is now known in advance, and the instance waits up to 10 minutes for
|
|
it before installing anything.
|
|
|
|
#### Azure
|
|
|
|
- Elastic and High Availability: redeploying the template always failed, so no parameter could be changed
|
|
after the first deployment:
|
|
- `InUseSubnetCannotBeDeleted`: the Master Nodes' subnets were declared outside the virtual network, so a
|
|
redeploy tried to delete them while the Master Nodes used them. Every subnet is now declared inside the
|
|
virtual network.
|
|
- `'Service Uri' must not be empty`: Azure Automation returns the URI of a webhook only when it creates it,
|
|
so the scale-in action group lost it on every redeploy. Every deployment now creates the scale-in webhook
|
|
under a new name. The webhooks of earlier deployments stay unused in the Automation account until they
|
|
expire.
|
|
|
|
A redeploy still cannot set the fixed, minimum or maximum number of Media Nodes below the Media Nodes
|
|
running (`UnsupportedCapacity`); the administration documentation describes how to remove them first.
|