How to get media streams for further processing?

Issues developing apps
1

Hi, How can I get media streams from sessions to pass them to ffmpeg or similar apps? This can be done reasonably easy with kurento, why this seems so hard with openvidu?

2

OpenVidu has the objective to provide a simpler API to build applications. Also, it provides other features like scalability and fault tolerance more difficult to provide if KMS API is provided as is.

In any case, Kurento Media Server is under the hood. Every OpenVidu session is backed by a MediaPipeline in a media node. There is no way in the public API to know what is the id of the MediaPipeline of an OpenVidu session, but you can tweak a bit the code to get it.

Take into account that we do not provide support for accessing directly to the Media Pipeline and the code can change without notice and break your app if you access directly to it.

Best regards

3

@micael.gallego Thanks a lot for your response and guiding me to the related openvidu code. Just another question, can I access and modify that mediapipline from another application? for example using a node application that talks to kurento, getting the pipline and then add an rtpEndpoint. Or do I have to fork openvidu and change its codes for desired result?

4

You can modify the internal Kurento Media Pipeline from any other process if you want. But make sure that the internal status of openvidu-server remains coherent and cohesive. What I mean is that if you want to get some information of the Media Pipeline, export it to an external endpoint not related to OpenVidu or modify it at a low-level in some way, that should be fine. But if you destroy Media Elements or re-connect them without letting OpenVidu Server know, then you will encounter problems for sure, as both processes (KMS and OpenVidu Server) will have different information about what’s going on in the Media Pipeline.

5

Thanks pablo, if I understand correctly, reconnecting current media elements used by openvidu (like WebrtcEndpoint) will cause problems, right? What about adding a new RtpEndpoint and connect it to these WebrtcEndpoints? Will this action also cause problems?
I want to capture webrtc streams and send them to ffmpeg so I need a RtpEndpoint to be able to do this job.

6

If you carefully take care of the lifecycle of the RtpEndpoint and properly respond to OpenVidu actions upon the WebrtcEndpoint (if OpenVidu is closing the WebRtcEndpoint, you should close the affected RtpEndpoint), then you should be able to do such thing.

7

Ok I see, thanks again

8

@pabloFuente , @micael.gallego I’m reading openvidu codes and trying to figure out how to add an rtpEndpoint and connect it to webrtcendpoint. Could you please guide me how and where I can do this?
So far I understand that there are three types of mediaEndpoint, webrtcEndpoint, playerEndpoint and rtpEndpoint. I think set endpointType to something other than webrtc I can create an rtpEndpoint. And publisher object is created in KurentoParticipant.java file inside createPublishingEndpoint, is that right? But I cannot understand where to create rtpEndpoint and how to connect it to webrtcEndpoint. I really appreciate any

9

I’m sorry but we never tried to add RtpEndpoint to OpenVidu sessions.

You will have to understand the code yourself and test different strategies.

I recommend you to first understand how Kurento works and then take a look to OpenVidu source code.

Regards

10

Thanks micael, I think I understand kurento and have done a few works with its node client. But I don’t understand the workflow of openvidu and where the webrtcEndpoints are created and managed. Could you at least recommend me how to approach the openvidu source code or give me a brief overview of its code structure?

11

I can tell you that everything related to WebRtcEndpoint objects is located in this folder of openvidu-server project: openvidu/openvidu-server/src/main/java/io/openvidu/server/kurento/endpoint at master · OpenVidu/openvidu · GitHub

That’s a good starting point to understand how openvidu-server manages and connects Kurento WebRtcEndpoints.

1 Like
12

I use web hooks in open vidu to send notification to backend program that looks to see if stream ended and then downloads the recording from the vidu server, processes it with ffmpeg and then sends a final api request back to vidu to delete the original