IP camera support using OpenVidu with Mediasoup backend

OpenVidu v2 Issues developing apps
1

Hello,

I have a working OpenVidu installation that is using MediaSoup backend (enterprise license).

All my attempts to stream IP cameras fail with an error.

Steps to reproduce:

  1. Use a test RTSP stream that works e.g. with VLC player:
    rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4

  2. Start a session and check session ID via Inspector. Say, “ses_KJnhexEjw3”

  3. Use curl to add RTSP stream to this session, following the example in OpenVidu API documentation.

Run in terminal:
> curl -X POST -H "Content-Type: application/json" -H "Authorization: Basic MY_SECRET" --data '{"type":"IPCAM","data":"BigBugBunny","record":true,"rtspUri":"rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4","adaptativeBitrate": true,"onlyPlayWithSubscribers":true,"networkCache":2000}' https://MY_SERVER/openvidu/api/sessions/ses_KJnhexEjw3/connection

Response from the server:
{“timestamp”:1645013880705,“status”:500,“error”:“Internal Server Error”,“message”:“Unable to create publisher endpoint. Code: 303”,“path”:"/openvidu/api/sessions/ses_KJnhexEjw3/connection"}

  1. Observe that new client appears to Inspector and also to OpenVidu session UI among other pariticipants. But without video stream.

  2. Check OpenVidu server logs:

openvidu-server_1  | [INFO] 2022-02-16 12:18:00,652 [http-nio-0.0.0.0-5443-exec-7] io.openvidu.server.rest.SessionRestController - REST API: POST /openvidu/api/sessions/ses_KJnhexEjw3/connection {type=IPCAM, data=BigBugBunny, record=true, rtspUri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4, adaptativeBitrate=true, onlyPlayWithSubscribers=true, networkCache=2000}
openvidu-server_1  | [INFO] 2022-02-16 12:18:00,678 [http-nio-0.0.0.0-5443-exec-7] io.openvidu.server.coturn.CoturnCredentialsService - Creating COTURN user
openvidu-server_1  | [INFO] 2022-02-16 12:18:00,697 [http-nio-0.0.0.0-5443-exec-7] io.openvidu.server.coturn.CoturnCredentialsService - COTURN user created: true
openvidu-server_1  | [INFO] 2022-02-16 12:18:00,697 [http-nio-0.0.0.0-5443-exec-7] io.openvidu.server.kurento.core.KurentoSession - SESSION ses_KJnhexEjw3: Added participant [participantPrivateId=ipc_IPCAM_rtsp_NDKU_wowzaec2demo_streamlock_net_vod_mp4_BigBuckBunny_115k_mp4, participantPublicId=ipc_IPCAM_rtsp_NDKU_wowzaec2demo_streamlock_net_vod_mp4_BigBuckBunny_115k_mp4, streaming=false]
openvidu-server_1  | [ERROR] 2022-02-16 12:18:00,704 [AbstractJsonRpcClientWebSocket-reqResEventExec-e219-t2628] io.openvidu.server.kurento.endpoint.PublisherEndpoint - EP str_IPC_QQSm_ipc_IPCAM_rtsp_NDKU_wowzaec2demo_streamlock_net_vod_mp4_BigBuckBunny_115k_mp4: Failed to create a new PlayerEndpoint
openvidu-server_1  | org.kurento.jsonrpc.JsonRpcErrorException: Error on 'create': [PlayerEndpoint.create] (PlayerEndpoint_30f41080-d463-453a-b46e-be9fbf5c6547) Method not implemented -- PlayerEndpoint is not yet available for use. Code: -1. Data: null

  1. Does this mean that streaming IP cameras is supported only with Kurento backend?

  2. If so, do you have plans to implement IP camera support with MediaSoup backend?

  3. If so, can you give even a rough estimate when this feature could be added, such as Q2/2022?

Thanks!

2

Yes. It should be documented in the OpenVidu Docs… but we forget it. We will update the docs for the 2.21.

It is not in our short term roadmap as we are focused in other stuff. Also, take into account that mediasoup lacks any form of processing camera video, so it is not possible to adapt the video to viewer bandwidth.

We will evaluate to include it in the mid term… maybe to Q3/2022 or Q4/2022.

Regards

3

Thank you for the quick reply, much appreciated!

I will elaborate a bit to give you some context: Our goal is a bit atypical, we are working to add different kinds of special cameras to OpenVidu based telcos as external camera feeds. As usual, a group of people joins the telco, and if necessary, they will add/invite a video feed from a special camera, which might be located somewhere else or carried by one of the participants. We currently support 360° cameras, PTZ security cameras, fixed and mobile thermal cameras, and wearable headsets. These can be controlled by telco participants, remotely, during the telco. Many of the supported cameras can join either via remote command or via showing an invite QR code to the camera.

In some cases our customers don’t care so much about latency of these camera feeds, but what is really important is the ability to see enough details and/or high enough frame rate. So in general we need much higher video quality than what is usually required for “talking heads on webcams”, but typically there is only one such camera in a telco. Lower quality is still OK for human users’ webcams.

We are already able to overcome many of the WebRTC video quality issues. For example, we can stream 120 Hz VGA video or 4K video at lower fps and use for example 15 Mbps bandwidth or more. However, it is a bit tricky to get such quality out of special cameras via WebRTC, and getting WebRTC to work on some camera models is also tricky. With some camera models, it would be much easier to input RTSP or RTMP stream on a server and then push this via WebRTC into the telco / browser clients.

This is why we are interested in direct RTSP (and RTMP) support. As it is not in your short term plans, we’ll probably consider using a custom server component as a mediator between certain camera models and OpenVidu/Mediasoup (as a workaround).