Hi,
we use OpenVidu (2.28) for streaming a live video from an self developed android app (from you example) to a self developed browser app (using openvidu browser).
Everything works fine - even the individual recording . with the exception of one point:
Recording fails, when the audio stream of the android app is disabled.
We have 2 methods in our app for disabling video or audio:
private fun toggleCamera(isEnabled: Boolean) {
Logger.info(if (isEnabled) "set camera to enabled" else "set camera to disabled")
localClient?.getVideoTrack()?.setEnabled(isEnabled)
callback.onCameraToggled(isEnabled)
}
private fun toggleMicrophone(isEnabled: Boolean) {
Logger.info(if (isEnabled) "set micro to enabled" else "set micro to disabled")
localClient?.getAudioTrack()?.setEnabled(isEnabled)
callback.onMicrophoneToggled(isEnabled)
}
toggleCamera → works well → recording works, even video track is disabled
toggleMicrophone → works well → recording works, just when audio track is enabled
If the audio track is disabled, I get the following error on the server:
Some RecorderEndpoint did not trigger RecordingEvent in time
[INFO] 2023-08-18 11:11:16,302 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.rest.SessionRestController - REST API: POST /openvidu/api/recordings/start {session=MY_APP_394, name=MY_APP_394, outputMode=INDIVIDUAL}
[INFO] 2023-08-18 11:11:16,302 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.RecordingManager - Getting free recording id for session MY_APP_394
[INFO] 2023-08-18 11:11:16,303 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.RecordingManager - Free recording id got for session MY_APP_394: MY_APP_394~1
[INFO] 2023-08-18 11:11:16,303 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.RecordingService - New recording id (MY_APP_394~1) and final name (MY_APP_394)
[INFO] 2023-08-18 11:11:16,303 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.SingleStreamRecordingService - Starting single stream recorder for stream str_MIC_M6vI_con_RbgA81wXxt in session MY_APP_394
[INFO] 2023-08-18 11:11:16,303 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.SingleStreamRecordingService - Starting individual (video+audio) recording MY_APP_394~1 of session MY_APP_394
[INFO] 2023-08-18 11:11:16,308 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.SingleStreamRecordingService - Starting single stream recorder for stream str_CAM_UWDY_con_U2G2syoupJ in session MY_APP_394
[INFO] 2023-08-18 11:11:16,354 [AbstractJsonRpcClientWebSocket-reqResEventExec-e2-t141] io.openvidu.server.recording.service.SingleStreamRecordingService - Recording started event for stream str_MIC_M6vI_con_RbgA81wXxt
[ERROR] 2023-08-18 11:11:26,313 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.RecordingService - Recording start failed for session MY_APP_394: Some RecorderEndpoint did not trigger RecordingEvent in time
[INFO] 2023-08-18 11:11:26,313 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.RecordingService - Sealed recording metadata file at /opt/openvidu/recordings/MY_APP_394~1/.recording.MY_APP_394~1 with status [failed]
[INFO] 2023-08-18 11:11:26,313 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.SingleStreamRecordingService - Stopping individual (video+audio) recording MY_APP_394~1 of session MY_APP_394. Reason: null
[INFO] 2023-08-18 11:11:26,314 [ForkJoinPool-4-worker-3] io.openvidu.server.recording.service.SingleStreamRecordingService - Stopping single stream recorder for stream str_MIC_M6vI_con_RbgA81wXxt in recording MY_APP_394~1
[INFO] 2023-08-18 11:11:26,314 [ForkJoinPool-4-worker-5] io.openvidu.server.recording.service.SingleStreamRecordingService - Stopping single stream recorder for stream str_CAM_UWDY_con_U2G2syoupJ in recording MY_APP_394~1
[INFO] 2023-08-18 11:11:26,318 [AbstractJsonRpcClientWebSocket-reqResEventExec-e2-t141] io.openvidu.server.recording.service.SingleStreamRecordingService - Recording stopped event for stream str_MIC_M6vI_con_RbgA81wXxt
[INFO] 2023-08-18 11:11:31,318 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.RecordingService - Sealed recording metadata file at /opt/openvidu/recordings/MY_APP_394~1/.recording.MY_APP_394~1 with status [failed]
[INFO] 2023-08-18 11:11:31,318 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.RecordingService - KMS recording file permissions successfully updated
[ERROR] 2023-08-18 11:11:31,314 [http-nio-0.0.0.0-5443-exec-4] io.openvidu.server.recording.service.SingleStreamRecordingService - Error waiting for some recorder endpoint to stop in session MY_APP_394
I am running out of ideas, what could be wrong??
@cruizba
I have some additional information, maybe you can tell me if these log entries can cause a problem or not:
→ set audioActive to false with our android application
[INFO] 2023-09-19 18:24:42,739 [SessionHandler-273khoalu0eaolsa6tn0lug0m-e8-t0] io.openvidu.server.rpc.RpcHandler - Participant con_BC0hwvtWSt is calling method 'onStreamPropertyChanged' in session openvidu_402
[INFO] 2023-09-19 18:24:42,754 [SessionHandler-273khoalu0eaolsa6tn0lug0m-e8-t0] io.openvidu.server.core.SessionEventsHandler - Video data of participant con_BC0hwvtWSt was initialized. height:720, width:1280, isVideoActive: true, isAudioActive: false
[INFO] 2023-09-19 18:24:44,902 [AbstractJsonRpcClientWebSocket-reqResEventExec-e2-t58] io.openvidu.server.kurento.core.KurentoParticipantEndpointConfig - KMS event [MediaFlowOutStateChanged] -> endpoint: str_CAM_CHek_con_BC0hwvtWSt (publisher) | state: NOT_FLOWING | pad: default | mediaType: AUDIO | timestamp: 1695147884902
[INFO] 2023-09-19 18:24:46,319 [AbstractJsonRpcClientWebSocket-reqResEventExec-e2-t58] io.openvidu.server.kurento.core.KurentoParticipantEndpointConfig - KMS event [MediaFlowInStateChanged] -> endpoint: con_VJXQwGbwdi_str_CAM_CHek_con_BC0hwvtWSt (subscriber) | state: NOT_FLOWING | pad: default | mediaType: AUDIO | timestamp: 1695147886318
→ set audioActive to false with our browser application
[INFO] 2023-09-19 18:32:53,039 [SessionHandler-uscrt1a2qio3hnfr3lotambpud-e16-t0] io.openvidu.server.rpc.RpcHandler - Participant con_HQikvbizrK is calling method 'onStreamPropertyChanged' in session openvidu_402
[INFO] 2023-09-19 18:32:53,087 [SessionHandler-uscrt1a2qio3hnfr3lotambpud-e16-t0] io.openvidu.server.core.SessionEventsHandler - Video data of participant con_HQikvbizrK was initialized. height:720, width:1280, isVideoActive: true, isAudioActive: false
As you can see, if I set audioActive to false with the android application, I got MediaFlowStateChanged-Events - but with the browser app not.
If you have to do a larger investigation - please contact me for making an offer for business support.
Best regards
Stefan
Hello,
I think that this is related to INDIVIDUAL recording not starting successfully when there is no audio or no video track to record. If you try to record an audio-only Stream with the audio track muted, the media server will try to start the recording process but no data will be received. The server’s response to this situation is relentless: if no publisher has published any media in the session trying to be recorded during a small timeout, the recording will fail. This is a decission that has pros and cons, and it is the default behavior.
This shouldn’t be a problem if you start the recording with the publisher’s strem unmuted, and then mute it. One way to adrress this problem is to try to start the recording. If that method returns this error, you know that the cause is that there is no media track to be recorded at that moment. Just try to start the recording again once there is at least some user sending at least an audio track to the server.
This behavior only affects INDIVIDUAL recordings. COMPOSED recordings do not behave like this.
Best regards.
Hi Pablo,
thanks for your answer! You are right, I think the behaviour of the Kurento server is completely ok. There is just no audio stream to record for the Kurento.
This shouldn’t be a problem if you start the recording with the publisher’s strem unmuted, and then mute it.
In this case, the recording works - but if I have a 30 seconds video recording - and I mute audio for 10 seconds - I end up with a 20 seconds audio - and video and audio are out if sync.
I think the problem is here:
localClient?.getAudioTrack()?.setEnabled(isEnabled)
From WebRTC specifications:
The enabled property on the MediaStreamTrack interface is a Boolean value which is true if the track is allowed to render the source stream or false if it is not. This can be used to intentionally mute a track.
When enabled, a track’s data is output from the source to the destination; otherwise, empty frames are output.
In the case of audio, a disabled track generates frames of silence (that is, frames in which every sample’s value is 0). For video tracks, every frame is filled entirely with black pixels.
I set the enabled property from AudioTrack to false - and WebRTC should generate frames of silence - but it just cancels the whole track.
With a browser app on the same device, it is working - but with the pure android app - i get this strange behaviour. Is it something your team could help by payed business support?
Or is it more a problem of androids webrtc framework and you could not help?
@pabloFuente
The error came from the compiled webrtc library in our android application.
We switched from
implementation("io.getstream:stream-webrtc-android:1.1.0")
to
implementation("com.dafruits:webrtc:120.0.0")
and the recording error is gone.
Best regards
Stefan
1 Like