Regular short freezes on MediaSoup

OpenVidu 2.20 Enterprise, deployed on AWS using Cloud Formation.

All working well. Mediasoup seems quick and good quality. I have one issue which is stuck at the moment:

Every minute or so, all video streams on all sessions pause very briefly. I have 6x active sessions, each with only 2x callers in. Every 1-2 minutes, the video on all these lock up for about 1-2 seconds. Each call is on a different machine.

I am pinging the AWS media node and master node every 0.1 seconds are there are no drops.

Any ideas?

Many thanks,

James

Hello,

Thank you for reporting the issue. We have also identified the behaviour.

We are investigating the issue and we expect to have it fixed in 2.21 version.

Regards

Thanks Micael.

Will you have any beta dockers available to try? I have a project active at the moment which is working well on OV Enterprise other than this issue.

Have you established the cause out of interest?

Thank you,

James

Hi @basicmonkey, could you please clarify a small but important detail of your feedback? You said:

Every minute or so, all video streams on all sessions pause very briefly. I have 6x active sessions, each with only 2x callers in. Every 1-2 minutes, the video on all these lock up for about 1-2 seconds. Each call is on a different machine.

So there are 6 independent sessions, each one on one of 6 separate machines (media nodes), and here is the important question:

  • All streams across all sessions freeze simultaneously every couple minutes?
  • Or each session freezes at a different point in time, regardless of the other sessions?

Also,

  • When the freeze occurs, it happens for all participants of the session at the same time? (in your case, it happens for the two participants)

Thanks

Hi @j1elo , thanks for looking into this!

It’s interesting. When I flagged this, I had 3x sessions across 2x machines (laptops and desktops), (each only 2x callers) all visible to me. The freezes were happening across all the sessions I could see, to both participants, at the same time. I also had 3x other sessions running which I could see one end of and these froze too.

Only running on 1x media node which is a c5.2xlarge. Nothing more than 5% CPU.

Today, I’m running a show on the same setup. The video quality is absolutely brilliant, best I’ve had with OV. The freezes now are only happening for some user pairs. Most of my calls are only one pair of users, today I have 2x groups of 12x people with a snoop machine to view only and I can see that today it’s all user dependant. Some users freeze, some users don’t.

In answer to your timing question, yesterday it was all together, today is is random. Some user pairs have no freezes. Some have regular ones, every 30-40 seconds.

My first test was on two high bandwidth networks, today is with end users on varied connections.

Many thanks,

James

Hi, after some investigation we found what could be a configuration issue with handling the retransmission feature, something that is important when there are lost packets in the network (a very common thing to happen).

I’ve just pushed an experimental new Docker image for the media node with a tentative fix, could you please test it and let us know if the fix improves things for your case?

The way to test this is to SSH into the OpenVidu master node, then

cd /opt/openvidu/

edit the .env file to add this line at the very end:

MEDIASOUP_IMAGE=openvidu/mediasoup-controller:2.21.0-dev3

exit the editor, and run this command:

./openvidu restart

In case you have OpenVIdu Enterprise High Availability deployed, follow this to change the content of your .env file:
https://docs.openvidu.io/en/stable/deployment/enterprise/aws/#2-change-configuration-by-modifying-s3-configuration-file-not-recommended

Hi j1elo.

Thanks for the beta. This seemed to be more stable but we still had intermittent freezes on calls that we replicated with our OV CE (KMS) server without any freezing.

It feels like Mediasoup is less inclined to let quality drop in order to maintain the stream.

Cheers,

James

It works for me
after added on.env “MEDIASOUP_IMAGE=openvidu/mediasoup-controller:2.21.0-dev3”
it is more stable.

my ov configure is follow
openvidu enterprise 2.20.1 -aws cloudformation
openvidu : c5.xlarge
kms: c5.xlarge

1 Like

Thanks for the feedback! We’ve applied the needed changes and will come with the next release of Openvidu.