We are using ionic 6.0 with capacitor 3.4.3 and everything works fine on web (even safari!) and android, but we are having issues with iOS (iPhone w/ ios 14.4.2) .
Our setup
We started by trying the example project, and it worked.
We even migrated the demo project to capacitor 3.4.2 to make sure that it worked.
Then we started the implementation following the guide: openvidu-ionic-capacitor - OpenVidu Docs
Here we had our first issues:
At one point the guide says to follow the “ios-specific-requirements”
but this guide is written for cordova, not capacitor.
This means that is not possible to use the provided hook that automatically set some configurations, so we had to do these steps manually as written here: cordova-plugin-iosrtc/Building.md at master · cordova-rtc/cordova-plugin-iosrtc · GitHub
Continuing with the guide we arrived at the step where we need to add “src/assets/libs/adapter-4.0.1.js”.
We added it, but we also noticed that in the example it is not used!
(Actually we found a commit that removed it in the example with cordova, but the application’s soruce code for capacitor is the same)
For sure the documentation could be improved/written for capacitor.
The issue:
It seems that this.session.publish(publisher) never completes. We don’t get any error, even with iosrtc’s debug mode enabled (cordova.plugins.iosrtc.debug.enable('*', true);)
We think that it might be some issue with the configuration of capacitor’s ios project because we had to guess some steps, but we are not sure.
We even tried to see the differences between the files generated by capacitor form out project and the example project but we could not find anything obviously…
These are the logs when we try to join a call:
[Debug] Connecting websocket to URI: wss://videocall-dev.ourwebsite/openvidu?sessionId=k31O3x3e0R (src_app_modules_videocall_videocall_module_ts.js, line 13335)
[Debug] WebSocket connected to wss://videocall-dev.ourwebsite/openvidu?sessionId=k31O3x3e0R (src_app_modules_videocall_videocall_module_ts.js, line 13335)
[Debug] --------- ONCONNECTED ----------- (src_app_modules_videocall_videocall_module_ts.js, line 13335)
[Debug] Starting ping (if configured) (src_app_modules_videocall_videocall_module_ts.js, line 13335)
[Debug] Sending request: {method:"joinRoom", params: {"token":"wss://videocall-dev.ourwebsite?sessionId=k31O3x3e0R&token=tok_NAw66WfMyFaBrSGr","session":"k31O3x3e0R","platform":"Safari 11.x on Apple iPhone (iOS 14.4.2)","sdkVersion":"2.21.0","metadata":"{\"role\":\"PUBLISHER\",\"screenShare\":false,\"externalUser\":false,\"user\":{\"id\":2,\"login\":\"user\",\"firstName\":\"User primo\",\"lastName\":\"User\",\"email\":\"user@localhost\",\"imageUrl\":\"http://localhost:8080/static/13f49257-4916-4f9c-90f1-88958f8e49b7.jpeg\",\"activated\":true,\"langKey\":\"it\",\"createdBy\":\"system\",\"createdDate\":null,\"lastModifiedBy\":\"system\",\"lastModifiedDate\":\"2022-05-30T10:00:04.959487Z\",\"authorities\":[\"ROLE_USER\"],\"birthDate\":null,\"residenceCity\":null,\"organization\":null,\"department\":null,\"profession\":null,\"url\":null,\"competences\":null,\"shortBiography\":null}}","secret":"","recorder":false}} (src_app_modules_videocall_videocall_module_ts.js, line 13335)
[Debug] Response: {"id":"con_YoFntyTyA5","finalUserId":"6681EB6E2EE3C82C","createdAt":1654780936742,"metadata":"{\"role\":\"PUBLISHER\",\"screenShare\":false,\"externalUser\":false,\"user\":{\"id\":2,\"login\":\"user\",\"firstName\":\"User primo\",\"lastName\":\"User\",\"email\":\"user@localhost\",\"imageUrl\":\"http://localhost:8080/static/13f49257-4916-4f9c-90f1-88958f8e49b7.jpeg\",\"activated\":true,\"langKey\":\"it\",\"createdBy\":\"system\",\"createdDate\":null,\"lastModifiedBy\":\"system\",\"lastModifiedDate\":\"2022-05-30T10:00:04.959487Z\",\"authorities\":[\"ROLE_USER\"],\"birthDate\":null,\"residenceCity\":null,\"organization\":null,\"department\":null,\"profession\":null,\"url\":null,\"competences\":null,\"shortBiography\":null}}%/%{\n \"userId\" : 2,\n \"externalNickname\" : null\n}","value":[],"session":"k31O3x3e0R","version":"2.20.0","mediaServer":"kurento","record":true,"role":"PUBLISHER","coturnIp":"40.68.222.82","turnUsername":"25CN8K","turnCredential":"ptz0lm","sessionId":"8run2lmn4qtpsu9mfuo8nv067p"} (src_app_modules_videocall_videocall_module_ts.js, line 13335)
[Info] openvidu-server version: 2.20.0 (src_app_modules_videocall_videocall_module_ts.js, line 13347)
[Error] openvidu-browser (2.21.0) and openvidu-server (2.20.0) versions are incompatible. Errors are likely to occur. openvidu-browser SDK is only compatible with the same version or the immediately following minor version of an OpenVidu deployment
[Info] 'Connection' created (local) (src_app_modules_videocall_videocall_module_ts.js, line 13347)
[Log] ios (src_app_modules_videocall_videocall_module_ts.js, line 607)
[Log] settings – {audioDevice: {kind: "audioinput", deviceId: "Built-In Microphone", label: "iPhone Microphone"}, videoDevice: {kind: "videoinput", deviceId: "com.apple.avfoundation.avcapturedevice.built-in_video:1", label: "Front Camera"}, enableMicrophone: true, …} (src_app_modules_videocall_videocall_module_ts.js, line 667)
{audioDevice: {kind: "audioinput", deviceId: "Built-In Microphone", label: "iPhone Microphone"}, videoDevice: {kind: "videoinput", deviceId: "com.apple.avfoundation.avcapturedevice.built-in_video:1", label: "Front Camera"}, enableMicrophone: true, enableWebcam: true, nickname: null}Object
[Log] publisher – Publisher {userHandlerArrowHandler: WeakMap, ee: EventEmitter, videos: [], …} (src_app_modules_videocall_videocall_module_ts.js, line 680)
Publisher {userHandlerArrowHandler: WeakMap, ee: EventEmitter, videos: [], lazyLaunchVideoElementCreatedEvent: false, stream: Stream, …}Publisher
[Log] iosrtc:getUserMedia [original constraints: (user-script:21, line 4632)
{audio: true, video: {width: {ideal: 640}, height: {ideal: 480}, frameRate: {ideal: 30}}}
] +18s
[Log] iosrtc:getUserMedia [computed constraints: (user-script:21, line 4632)
{video: {width: {ideal: 640}, height: {ideal: 480}, frameRate: {ideal: 30}}, audio: {}}
] +1ms
[Log] iosrtc:getUserMedia getUserMedia() | success +17ms (user-script:21, line 4632)
[Log] iosrtc:MediaStream create() | [dataFromEvent: (user-script:21, line 4632)
{id: "F58BA36F-2DE6-4539-9F75-5C2EF3F4DB9F", audioTracks: Object, videoTracks: Object}
] +17s
[Log] iosrtc:MediaStream new MediaStream(arg) | [arg: (user-script:21, line 4632)
[] (0)
] +0ms
[Log] iosrtc:MediaStreamTrack new() | [dataFromEvent: (user-script:21, line 4632)
{readyState: "live", id: "F01CBFDC-77BA-4757-9561-116FD883FC14", kind: "audio", enabled: true, capabilities: {}, …}
] +18s
[Log] iosrtc:MediaStreamTrack new() | [dataFromEvent: (user-script:21, line 4632)
{readyState: "live", id: "B3E25D55-A012-444B-8AA5-9F1C8F4E0283", kind: "video", enabled: true, capabilities: {deviceId: "com.apple.avfoundation.avcapturedevice.built-in_video:1"}, …}
] +1ms
[Log] iosrtc:MediaStream emitConnected() +3ms (user-script:21, line 4632)
[Log] iosrtc:MediaStream getAudioTracks() +1ms (user-script:21, line 4632)
[Log] iosrtc:MediaStream getAudioTracks() +0ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamTrack enabled = true +1ms (user-script:21, line 4632)
[Log] iosrtc:MediaStream getVideoTracks() +0ms (user-script:21, line 4632)
[Log] iosrtc:MediaStream getVideoTracks() +1ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamTrack enabled = false +1ms (user-script:21, line 4632)
[Log] iosrtc:MediaStream getAudioTracks() +0ms (user-script:21, line 4632)
[Info] Audio track Content Hint set: '' (src_app_modules_videocall_videocall_module_ts.js, line 13347)
[Log] iosrtc:MediaStream getVideoTracks() +0ms (user-script:21, line 4632)
[Info] Video track Content Hint set: 'motion' (src_app_modules_videocall_videocall_module_ts.js, line 13347)
[Debug] Video srcObject [F58BA36F-2DE6-4539-9F75-5C2EF3F4DB9F] updated in stream [undefined] (src_app_modules_videocall_videocall_module_ts.js, line 13335)
[Log] iosrtc:videoElementsHandler new video element added +17s (user-script:21, line 4632)
[Log] iosrtc:videoElementsHandler observeVideo() +0ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer new() | [element:"<video>"] +17s (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer refresh() +2ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer refresh() | no video track yet +0ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer refresh() | [data: (user-script:21, line 4632)
{elementLeft: 0, elementTop: 0, elementWidth: 0, elementHeight: 0, videoViewWidth: 0, …}
] +0ms
[Log] iosrtc:MediaStreamRenderer render() [stream: (user-script:21, line 4632)
MediaStream {, …}
] +2ms
[Log] iosrtc:MediaStreamRenderer close() +3ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer new() | [element:"<video>"] +0ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer refresh() +1ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer refresh() | no video track yet +0ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer refresh() | [data: (user-script:21, line 4632)
{elementLeft: 0, elementTop: 0, elementWidth: 0, elementHeight: 0, videoViewWidth: 0, …}
] +0ms
[Log] iosrtc:MediaStreamRenderer render() [stream: (user-script:21, line 4632)
MediaStream {, …}
] +0ms
[Log] iosrtc:MediaStreamRenderer onEvent() | [type:videoresize, data: (user-script:21, line 4632)
{size: {width: 480, height: 640}, type: "videoresize"}
] +184ms
[Log] iosrtc:MediaStreamRenderer refresh() +1ms (user-script:21, line 4632)
[Log] iosrtc:MediaStreamRenderer refresh() | [data: (user-script:21, line 4632)
{elementLeft: 32, elementTop: 93, elementWidth: 311, elementHeight: 311, videoViewWidth: 311, …}
] +0ms
Yes, we are aware of
[Error] openvidu-browser (2.21.0) and openvidu-server (2.20.0) versions are incompatible
but right now it’s difficult for us to update the openvidu-server or downgrade openvidu-browser. We think/hope that the problem is not related to this. On the other plafroms (android/web we didn’t have any issue). In the next days we might try to upgrade openvidu-browser to 2.22 and try to connect to openvidu’s demo server that it’s also on 2.22
Do you have any hint on where the issue could be? Or suggestions on how we can debug why this.session.publish(publisher) never completes?
Thanks a lot!
