Hi,
I experienced the same error several times. I am using the latest OpenVidu 2.16.0 with standard docker installation on premise(So Kurento Media Server 6.15.0).
Somehow(i don’t know the reason) docker container on which KMS runs (openvidu_kms_1) restarts itself. After this unexpected restart I get the error “There is no available Media Node where to initialize session Code: 204” from OpenVidu Server while trying to initiate sessions. Thus, I think that OpenVidu server cannot connect KMS after this restart.
I tried to restart only “openvidu_openvidu-server_1” docker container by “docker container restart openvidu_openvidu-server_1” , everything became normal again and I was able to initiate sessions as expected.
Here are the logs from KMS when unexpected restart occurs. It can be seen that logs stuck at " Kurento Media Server started":
2020-12-18T08:44:22,191810 1 0x00007f5826a46700 info basertpendpoint kmsbasertpendpoint.c:1064 kms_base_rtp_endpoint_create_remb_manager() <kmswebrtcendpoint11603> Creating REMB for session ID 0 (kmswebrtcendpoint11603-sess0) and remote video SSRC 0 2020-12-18T08:44:22,191843 1 0x00007f5826a46700 info basertpendpoint kmsbasertpendpoint.c:1075 kms_base_rtp_endpoint_create_remb_manager() <kmswebrtcendpoint11603> REMB: Set RTCP min interval to 500 ms 2020-12-18T08:44:22,308155 1 0x00007f5828a4a700 info KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:108 remove_not_supported_codecs_from_array() <kmswebrtcendpoint11604> Removing not supported codec 'AMR/8000' 2020-12-18T08:44:22,308320 1 0x00007f5828a4a700 info KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:566 WebRtcEndpointImpl() STUN port not found in config; using default value: 3478 2020-12-18T08:44:22,308367 1 0x00007f5828a4a700 info KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:574 WebRtcEndpointImpl() STUN server not found in config; remember that NAT traversal requires STUN or TURN 2020-12-18T08:44:22,308401 1 0x00007f5828a4a700 info KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:597 WebRtcEndpointImpl() TURN server not found in config; remember that NAT traversal requires STUN or TURN 2020-12-18T08:44:22,318942 1 0x00007f5523686700 fixme basesink gstbasesink.c:3125 gst_base_sink_default_event() <nicesink11601> stream-start event without group-id. Consider implementing group-id handling in the upstream elements 2020-12-18T08:44:22,319243 1 0x00007f5519e73700 fixme default gstutils.c:3766 gst_pad_create_stream_id_internal() <nicesrc11601:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id 2020-12-18T08:44:22,320156 1 0x00007f5826245700 info basertpendpoint kmsbasertpendpoint.c:1118 kms_base_rtp_endpoint_start_transport_send() <kmswebrtcendpoint11604> Media 'video' has REMB 2020-12-18T08:44:22,320205 1 0x00007f5826245700 info basertpendpoint kmsbasertpendpoint.c:1064 kms_base_rtp_endpoint_create_remb_manager() <kmswebrtcendpoint11604> Creating REMB for session ID 0 (kmswebrtcendpoint11604-sess0) and remote video SSRC 0 2020-12-18T08:44:22,320255 1 0x00007f5826245700 info basertpendpoint kmsbasertpendpoint.c:1075 kms_base_rtp_endpoint_create_remb_manager() <kmswebrtcendpoint11604> REMB: Set RTCP min interval to 500 ms 2020-12-18T08:44:24,458817 1 0x00007f55a3f87700 info kmsutils kmsutils.c:515 kms_utils_pad_monitor_gaps() <'':sink_video_default> Add probe: DISCONT buffers and GAP events 2020-12-18T08:44:24,589316 1 0x00007f55a4788700 info kmsutils kmsutils.c:515 kms_utils_pad_monitor_gaps() <'':sink_video_default> Add probe: DISCONT buffers and GAP events 2020-12-18T08:44:25,651095 1 0x00007f13c6783240 info KurentoMediaServer main.cpp:196 main() Logs storage path set to /opt/openvidu/kurento-logs 2020-12-18T08:44:25,651727 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:90 loadModule() Load file: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmscoremodule.so, module name: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmscoremodule.so 2020-12-18T08:44:25,651764 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:137 loadModule() Loaded module: core, version: 6.15.0, date: Nov 3 2020 18:12:59 2020-12-18T08:44:25,652339 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:90 loadModule() Load file: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmsfiltersmodule.so, module name: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmsfiltersmodule.so 2020-12-18T08:44:25,652367 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:137 loadModule() Loaded module: filters, version: 6.15.0, date: Nov 3 2020 18:33:58 2020-12-18T08:44:25,657335 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:90 loadModule() Load file: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmselementsmodule.so, module name: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmselementsmodule.so 2020-12-18T08:44:25,657383 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:137 loadModule() Loaded module: elements, version: 6.15.0, date: Nov 3 2020 18:22:58 2020-12-18T08:44:25,657802 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:90 loadModule() Load file: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmspointerdetectormodule.so, module name: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmspointerdetectormodule.so 2020-12-18T08:44:25,657825 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:137 loadModule() Loaded module: pointerdetector, version: 6.15.0, date: Nov 4 2020 12:26:22 2020-12-18T08:44:25,658208 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:90 loadModule() Load file: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmsplatedetectormodule.so, module name: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmsplatedetectormodule.so 2020-12-18T08:44:25,658234 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:137 loadModule() Loaded module: platedetector, version: 6.15.0, date: Nov 4 2020 12:26:25 2020-12-18T08:44:25,658689 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:90 loadModule() Load file: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmscrowddetectormodule.so, module name: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmscrowddetectormodule.so 2020-12-18T08:44:25,658714 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:137 loadModule() Loaded module: crowddetector, version: 6.15.0, date: Nov 4 2020 12:26:52 2020-12-18T08:44:25,659102 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:90 loadModule() Load file: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmschromamodule.so, module name: /usr/lib/x86_64-linux-gnu/kurento/modules/libkmschromamodule.so 2020-12-18T08:44:25,659128 1 0x00007f13c6783240 info KurentoModuleManager ModuleManager.cpp:137 loadModule() Loaded module: chroma, version: 6.15.0, date: Nov 4 2020 12:20:06 2020-12-18T08:44:25,659162 1 0x00007f13c6783240 info KurentoMediaServer main.cpp:240 main() Kurento Media Server version: 6.15.0 2020-12-18T08:44:25,659175 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:249 loadConfig() Reading configuration from: /etc/kurento/kurento.conf.json 2020-12-18T08:44:25,659488 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:183 loadModulesConfigFromDir() Looking for config files in /etc/kurento/modules 2020-12-18T08:44:25,659606 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:183 loadModulesConfigFromDir() Looking for config files in /etc/kurento/modules/kurento 2020-12-18T08:44:25,659782 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:209 loadModulesConfigFromDir() Loaded module config: /etc/kurento/modules/kurento/BaseRtpEndpoint.conf.ini 2020-12-18T08:44:25,660002 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:209 loadModulesConfigFromDir() Loaded module config: /etc/kurento/modules/kurento/HttpEndpoint.conf.ini 2020-12-18T08:44:25,660114 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:209 loadModulesConfigFromDir() Loaded module config: /etc/kurento/modules/kurento/MediaElement.conf.ini 2020-12-18T08:44:25,660267 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:209 loadModulesConfigFromDir() Loaded module config: /etc/kurento/modules/kurento/WebRtcEndpoint.conf.ini 2020-12-18T08:44:25,660396 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:209 loadModulesConfigFromDir() Loaded module config: /etc/kurento/modules/kurento/UriEndpoint.conf.ini 2020-12-18T08:44:25,660613 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:209 loadModulesConfigFromDir() Loaded module config: /etc/kurento/modules/kurento/SdpEndpoint.conf.json 2020-12-18T08:44:25,660748 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:209 loadModulesConfigFromDir() Loaded module config: /etc/kurento/modules/kurento/PlayerEndpoint.conf.ini 2020-12-18T08:44:25,660781 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:265 loadConfig() Configuration loaded successfully 2020-12-18T08:44:25,660852 1 0x00007f13c6783240 info KurentoLoadConfig loadConfig.cpp:271 loadConfig() Loaded config in effect: { "mediaServer": { "resources": { "\/\/exceptionLimit": "0.8", "\/\/killLimit": "0.7", "garbageCollectorPeriod": "240", "\/\/": "Default: false", "disableRequestCache": "false" }, "net": { "websocket": { "\/\/ipv6": "false", "port": "8888", "secure": { "\/\/port": "8433", "\/\/certificate": "cert+key.pem", "\/\/": "Password for the private key, if one was set when the key was created", "\/\/password": "" }, "\/\/registrar": { "\/\/address": "ws:\/\/localhost:9090", "\/\/localAddress": "localhost" }, "\/\/": "Default: SOMAXCONN (128)", "\/\/connqueue": "128", "path": "kurento", "threads": "10" } } }, "configPath": "\/etc\/kurento", "modules": { "kurento": { "BaseRtpEndpoint": { "minPort": "40000", "maxPort": "57000", "configPath": "\/etc\/kurento\/modules\/kurento" }, "HttpEndpoint": { "serverAddress": "localhost", "port": "9091", "configPath": "\/etc\/kurento\/modules\/kurento" }, "MediaElement": { "configPath": "\/etc\/kurento\/modules\/kurento" }, "WebRtcEndpoint": { "configPath": "\/etc\/kurento\/modules\/kurento" }, "UriEndpoint": { "defaultPath": "file:\/\/\/var\/lib\/kurento\/", "configPath": "\/etc\/kurento\/modules\/kurento" }, "SdpEndpoint": { "numAudioMedias": "1", "numVideoMedias": "1", "audioCodecs": [ { "name": "opus\/48000\/2", "\/\/": "Next is an example about how a codec can be configured.", "\/\/": "WARNING: Usage of properties is not yet supported", "\/\/properties": { "\/\/maxcodedaudiobandwidth": "16000", "\/\/maxaveragebitrate": "20000", "\/\/stereo": "1", "\/\/useinbandfec": "1", "\/\/usedtx": "0" } }, { "name": "PCMU\/8000" }, { "name": "AMR\/8000" } ], "videoCodecs": [ { "name": "VP8\/90000" }, { "name": "H264\/90000" } ], "configPath": "\/etc\/kurento\/modules\/kurento" }, "PlayerEndpoint": { "configPath": "\/etc\/kurento\/modules\/kurento" } } } } 2020-12-18T08:44:25,660902 1 0x00007f13c6783240 info KurentoServerMethods ServerMethods.cpp:90 ServerMethods() Using above 80% of system limits will throw NOT_ENOUGH_RESOURCES exception 2020-12-18T08:44:25,660924 1 0x00007f13c6783240 info KurentoServerMethods ServerMethods.cpp:109 ServerMethods() System limits: unlimited threads, 1048576 files 2020-12-18T08:44:25,661131 1 0x00007f13c6783240 info KurentoWorkerPool WorkerPool.cpp:67 WorkerPool() Worker thread pool size: 16 2020-12-18T08:44:25,661678 1 0x00007f13c6783240 info KurentoServerMethods ServerMethods.cpp:144 ServerMethods() RPC Request Cache is ENABLED 2020-12-18T08:44:25,661794 1 0x00007f13c6783240 info KurentoWebSocketTransport WebSocketTransport.cpp:179 initWebSocket() WebSocket server (ws://) listening on address '::', port 8888 2020-12-18T08:44:25,661813 1 0x00007f13c6783240 info KurentoWebSocketTransport WebSocketTransport.cpp:88 WebSocketTransport() Secure WebSocket server (wss://) not enabled 2020-12-18T08:44:25,662067 1 0x00007f13c6783240 info KurentoMediaServer main.cpp:259 main() Kurento Media Server started
When I restarted only “openvidu_openvidu-server_1” log flow continued. Normal behaviour of OpenVidu returns.
I want to ask what might be the reason behind this unexpected restart of KMS and what should I do make sure that at least openvidu connects again KMS after restart? What further steps should I follow?
Thanks.