Session can be broken with an Android client

trantako · May 17, 2022, 1:32pm

Hello,

We have observed a case where an Android client can break the whole session by attempting to publish video with a slightly faulty configuration.

We used openvidu-android client as the basis for our app:

We made some customisations/optimisations. One of them was using hardware video encoder. However, when we tested the app with an Android phone that does not have HW video encoder, the following happened:

In Android client, creating local description succeeded but setting it as the local description failed.
The created SDP was missing necessary fields. For example, there was
m=video 9 UDP/TLS/RTP/SAVPF 0
, while a successful SDP contains something like
m=video 9 UDP/TLS/RTP/SAVPF 96 100
, and there are also more lines that describe 96 and 100 as video codec options.
Despite of the failure, the example code proceeded to attempt to publish video.
The client appeared in the session, without video. However, it also caused the following:

various errors in server and client logs
every participant’s video streams got stuck
the session could not be properly closed (it became a ghost session, required OpenVidu server reboot to clear it)

Client errors:

code -1: org.kurento.client.internal.server.KurentoServerException:Connection with server have been closed
code 303: Unable to create subscriber endpoint. Code: 303

Server errors:

[nioEventLoopGroup-3-1] org.kurento.jsonrpc.client.JsonRpcClientNettyWebSocket -  channel closed
[AbstractJsonRpcClientWebSocket-disconnectExec-e4-t0] org.kurento.jsonrpc.client.AbstractJsonRpcClientWebSocket -  Exception trying to reconnect to server ws://localhost:8888/kurento. Notifying disconnection
org.kurento.jsonrpc.JsonRpcException:  Exception connecting to WebSocket server ws://localhost:8888/kurento
openvidu-server_1  | 	at org.kurento.jsonrpc.client.AbstractJsonRpcClientWebSocket.internalConnectIfNecessary(AbstractJsonRpcClientWebSocket.java:735)
openvidu-server_1  | 	at org.kurento.jsonrpc.client.AbstractJsonRpcClientWebSocket.connectIfNecessary(AbstractJsonRpcClientWebSocket.java:847)
openvidu-server_1  | 	at org.kurento.jsonrpc.client.AbstractJsonRpcClientWebSocket$16.run(AbstractJsonRpcClientWebSocket.java:635)
openvidu-server_1  | 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
openvidu-server_1  | 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
openvidu-server_1  | 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
openvidu-server_1  | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
openvidu-server_1  | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
openvidu-server_1  | 	at java.base/java.lang.Thread.run(Thread.java:829)
openvidu-server_1  | Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8888
openvidu-server_1  | Caused by: java.net.ConnectException: Connection refused
openvidu-server_1  | 	at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
openvidu-server_1  | 	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
openvidu-server_1  | 	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
openvidu-server_1  | 	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
openvidu-server_1  | 	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:707)
openvidu-server_1  | 	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
openvidu-server_1  | 	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
openvidu-server_1  | 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
openvidu-server_1  | 	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
openvidu-server_1  | 	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
openvidu-server_1  | 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
openvidu-server_1  | 	at java.base/java.lang.Thread.run(Thread.java:829)

Etc. etc.

We have created a fix for the Android client, which should check if setting local description has failed:

github.com/OpenVidu/openvidu-tutorials

Fixed an attempt to publish video after failure to set the local desc…

OpenVidu:master ← FinweLtd:android_client_fixes

opened 12:59PM - 17 May 22 UTC

trantako

+16 -2

When preparing to publish a video stream from an Android client app to OpenVidu …session, it is possible that creating video offer succeeds but then setting it as the local description fails. An example: Android client is configured to use a hardware video encoder, but a particular device does not have one. As a result, SDP description is created without some necessary video elements, and setting it as the local description fails at runtime. The original code did not check the result of setting a local description before it went on to attempt publishing video. This resulted to breaking the whole OpenVidu session with rather severe consequences: - various errors in server and client logs - every participant's video streams stuck - the session cannot be properly closed (it becomes a ghost session) This simple fix checks if setting the local description succeeds, and only then attempts to publish video stream. While this demo application does not configure HW encoder in use, other developers will use it as a basis for their apps. In addition, there can be other reasons why setting local description may fail. Hence, it should be checked.

However, a single client should not be able to crash a complete session like that. The server side code probably needs modifications, too.

Here’s an example of an SDP from an attempt to publish video that resulted to session crash. The client claims that it will publish video, but the SDP does not have any video codecs:

    v=0
    o=- 7719772216287633120 2 IN IP4 127.0.0.1
    s=-
    t=0 0
    a=group:BUNDLE 0 1
    a=msid-semantic: WMS
    m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126
    c=IN IP4 0.0.0.0
    a=rtcp:9 IN IP4 0.0.0.0
    a=ice-ufrag:c2WV
    a=ice-pwd:GZ7nmy4N6x4Ca63YsFIPmJjj
    a=ice-options:trickle renomination
    a=fingerprint:sha-256 [REMOVED]
    a=setup:actpass
    a=mid:0
    a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
    a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
    a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
    a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
    a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
    a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
    a=sendonly
    a=msid:- 101
    a=rtcp-mux
    a=rtpmap:111 opus/48000/2
    a=rtcp-fb:111 transport-cc
    a=fmtp:111 minptime=10;useinbandfec=1
    a=rtpmap:103 ISAC/16000
    a=rtpmap:104 ISAC/32000
    a=rtpmap:9 G722/8000
    a=rtpmap:102 ILBC/8000
    a=rtpmap:0 PCMU/8000
    a=rtpmap:8 PCMA/8000
    a=rtpmap:106 CN/32000
    a=rtpmap:105 CN/16000
    a=rtpmap:13 CN/8000
    a=rtpmap:110 telephone-event/48000
    a=rtpmap:112 telephone-event/32000
    a=rtpmap:113 telephone-event/16000
    a=rtpmap:126 telephone-event/8000
    a=ssrc:2314895369 cname:9Vpk6URscoMETSuX
    a=ssrc:2314895369 msid:- 101
    a=ssrc:2314895369 mslabel:-
    a=ssrc:2314895369 label:101
    m=video 9 UDP/TLS/RTP/SAVPF 0
    c=IN IP4 0.0.0.0
    a=rtcp:9 IN IP4 0.0.0.0
    a=ice-ufrag:c2WV
    a=ice-pwd:[REMOVED]
    a=ice-options:trickle renomination
    a=fingerprint:sha-256 [REMOVED]
    a=setup:actpass
    a=mid:1
    a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
    a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
    a=extmap:13 urn:3gpp:video-orientation
    a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
    a=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
    a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
    a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
    a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
    a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
    a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
    a=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
    a=sendonly
    a=msid:- 100
    a=rtcp-mux
    a=rtcp-rsize
    a=ssrc:4108384921 cname:9Vpk6URscoMETSuX
    a=ssrc:4108384921 msid:- 100
    a=ssrc:4108384921 mslabel:-
    a=ssrc:4108384921 label:100

pabloFuente · May 18, 2022, 9:12am

Hi,

Thank you for your report. I agree: a wrong SDP on the publish operation shouldn’t break the session. Let’s keep track of this issue in GitHub from now on. Your PR is a good place to do so: Fixed an attempt to publish video after failure to set the local desc… by trantako · Pull Request #174 · OpenVidu/openvidu-tutorials · GitHub

pabloFuente · May 18, 2022, 11:29am

I have also opened this official issue: Session crashes when processing malformed SDP · Issue #724 · OpenVidu/openvidu · GitHub

Topic		Replies	Views
Android client crahes whole session Issues developing apps v2	3	316	October 13, 2020
Not able close session when GStreamer is enabled Issues developing apps v2	0	90	December 11, 2023
Timeout connecting to session Issues developing apps v2	0	230	February 21, 2022
Error setting min/max bandwidth for endpoint Issues with deployment v2	1	478	February 1, 2021
All connections suddenly dropped Issues developing apps v2	7	470	November 28, 2020

Session can be broken with an Android client

Related topics