openvidu/openvidu-deployment/CHANGELOG.md

Changelog

Every release groups its Added, Improved and Fixed entries by the cloud they apply to:

• All clouds: changes that behave the same on every cloud.
• AWS, Azure, Google Cloud, Oracle Cloud and DigitalOcean: what is specific to that cloud's templates.

The AWS, Azure and Google Cloud templates live in this folder. The Oracle Cloud and DigitalOcean templates live in the openvidu-oracle and openvidu-digitalocean repositories.

3.10.0 (unreleased)

Added

All clouds

• Elastic and High Availability deployments replace an unhealthy Media Node with a new one automatically, keeping the same number of Media Nodes. Before, a broken Media Node stayed in the cluster until it was replaced by hand. Every Media Node runs a health watchdog (the openvidu-media-health systemd service) that checks its media server every 30 seconds and asks the cloud to replace the node when:
  • the media server does not accept connections for 5 minutes,
  • the media server answers with errors or timeouts for 10 minutes (not counted while the node's CPU is saturated, because an overloaded media server briefly reports itself as not ready),
  • the node's installation failed (after a 10 minute wait, so the failure can be inspected and a persistent problem does not create new nodes in a loop),
  • or the node's installation has not finished 90 minutes after it booted.
• Safeguards of the Media Node watchdog:
  • No Media Node is replaced while no Master Node is reachable: a Master Node outage makes every Media Node fail its health check at once, and a new node could not join the cluster either. Once a Master Node is back, the media server gets its full time window to reconnect before the watchdog acts.
  • Failures are not counted during the first 15 minutes after OpenVidu (re)starts on the node.
  • The watchdog pauses itself while a node is drained during a normal scale-in.
  • journalctl -u openvidu-media-health on a Media Node shows what it is doing, and sudo touch /etc/openvidu/media-health.disabled pauses it (for example during manual maintenance); deleting the file resumes it.

AWS

• An unhealthy Media Node is marked unhealthy in its Auto Scaling Group, which drains it and launches a new one.

Azure

• An unhealthy Media Node is reimaged in its Virtual Machine Scale Set, so the number of instances never changes.

Google Cloud

• An unhealthy Media Node is recreated in its managed instance group with a fresh disk. The group's native autohealing (TCP health check on port 7880), now also enabled in High Availability deployments, covers a Media Node VM that stops responding altogether.

Oracle Cloud

• An unhealthy Media Node is terminated and its instance pool launches a new one. With a fixed number of Media Nodes, the node is detached from the pool, which launches its replacement, and then terminated.

DigitalOcean

• With autoscaling, an unhealthy Media Node is tagged so the autoscaler deletes it and creates a new one. With a fixed number of Media Nodes, the Droplet is rebuilt in place.

Fixed

All clouds

• High Availability: publishing to an RTMP ingress failed. OpenVidu returns the ingress URL on port 1945 (rtmps://<your-domain>:1945/rtmp), but the load balancer only accepted RTMP on port 1935. The load balancer now accepts RTMPS on port 1945. Publish to the URL OpenVidu returns when the ingress is created, and allow port 1945 in any firewall in front of your publishers.

AWS

• Single node and Elastic: the Elastic IP was attached only after the instance had started booting, so the domain could be derived from a temporary public IP, or the instance could start without Internet access and fail its installation. The Elastic IP is now known in advance, and the instance waits up to 10 minutes for it before installing anything.