In my app, publishing and unpublishing happens several times dynamically. If the session.publish method is called while the browser window is not active (user is looking at some other app on their computer), the promise doesn’t complete until the user activates the browser window again. When the user activates the window, the publish method completes causing the camera to become engaged. This causes issues in my app because when the user activates the window it may be after the app has already automatically unpublished them. This also results in several errors such as org.kurento.client.internal.server.KurentoServerException:Endpoint already negotiated.
Do you have any suggestions for dealing with this? Possibly a way to cancel a previous call to publish? Calling unpublish on a publisher doesn’t seem to cancel the unfinished call to publish.
It is my own app. In order to reproduce this, you would have to have some way to call publish while the browser window is not active. Maybe a simple call to setTimeout would work. Something like this…
start 2 timers; one to publish in 5 seconds, one to unpublish in 10 seconds
deactivate browser window
1st timer calls publish which doesn’t resolve because browser is not active (camera does not turn on)
2nd timer calls unpublish
reactivate browser window; this causes the publish method to now resolve and activates the camera, even though unpublish was already called.
EDIT: I’m working on a simple demo to share with you…
My app is not calling publish and unpublish at the same time.
My app is automatically calling publish while the browser is not active. This call never completes (the then or catch of the promise is never called) when the browser is not active. Then after about 10 seconds it calls unpublish. When I then active the browser window, the publish promise completes and the camera is turned on. I’m trying to prevent that because I have already called unpublish.
I’m working on a simple demo so I can show you what I mean.
I am able to reproduce this issue using the openvidu-hello-world demo with some modifications. The modified app.js file is below. The rest of the files were not changed. This is easiest to see when using a 2nd monitor, so you can have the browser on one monitor and then activate something on the other monitor while still being able to see the browser console.
Steps to reproduce the issue (I am using Firefox)…
load the app in the browser
open the browser console (I am logging when the methods are called and complete)
click the JOIN button and then activate another window (preferably on a different monitor)
after a few seconds you will see in the console that publish is called but never completes
after a few more seconds you will see that unpublish is called
now activate the browser window and you will see that the publish method completes, the camera becomes active, and there are a bunch of errors.
Thanks again.
// app.js
const OPENVIDU_SERVER_URL = "https://" + location.hostname + ":4443";
const OPENVIDU_SERVER_SECRET = "123";
let OV;
let session;
let publisher;
function joinSession() {
const mySessionId = document.getElementById("sessionId").value;
OV = new OpenVidu();
session = OV.initSession();
session.on("streamCreated", function (event) {
session.subscribe(event.stream, "subscriber");
});
getToken(mySessionId).then(token => {
session.connect(token)
.then(() => {
document.getElementById("session-header").innerText = mySessionId;
document.getElementById("join").style.display = "none";
document.getElementById("session").style.display = "block";
publish();
})
.catch(error => {
console.log("There was an error connecting to the session:", error.code, error.message);
});
});
}
function publish() {
window.setTimeout(() => {
console.log('*** calling publish');
publisher = OV.initPublisher("publisher");
session.publish(publisher).then(() => {
console.log('*** publishing');
}).catch(error => {
console.log('*** publish error', error);
});
}, 5000);
window.setTimeout(() => {
session.unpublish(publisher);
console.log('*** unpublished');
}, 10000);
}
function leaveSession() {
session.disconnect();
document.getElementById("join").style.display = "block";
document.getElementById("session").style.display = "none";
}
window.onbeforeunload = function () {
if (session) session.disconnect()
};
/**
* --------------------------
* SERVER-SIDE RESPONSIBILITY
* --------------------------
* These methods retrieve the mandatory user token from OpenVidu Server.
* This behavior MUST BE IN YOUR SERVER-SIDE IN PRODUCTION (by using
* the API REST, openvidu-java-client or openvidu-node-client):
* 1) Initialize a session in OpenVidu Server (POST /api/sessions)
* 2) Generate a token in OpenVidu Server (POST /api/tokens)
* 3) The token must be consumed in Session.connect() method
*/
function getToken(mySessionId) {
return createSession(mySessionId).then(sessionId => createToken(sessionId));
}
function createSession(sessionId) { // See https://docs.openvidu.io/en/stable/reference-docs/REST-API/#post-apisessions
return new Promise((resolve, reject) => {
$.ajax({
type: "POST",
url: OPENVIDU_SERVER_URL + "/api/sessions",
data: JSON.stringify({ customSessionId: sessionId }),
headers: {
"Authorization": "Basic " + btoa("OPENVIDUAPP:" + OPENVIDU_SERVER_SECRET),
"Content-Type": "application/json"
},
success: response => resolve(response.id),
error: (error) => {
if (error.status === 409) {
resolve(sessionId);
} else {
console.warn('No connection to OpenVidu Server. This may be a certificate error at ' + OPENVIDU_SERVER_URL);
if (window.confirm('No connection to OpenVidu Server. This may be a certificate error at \"' + OPENVIDU_SERVER_URL + '\"\n\nClick OK to navigate and accept it. ' +
'If no certificate warning is shown, then check that your OpenVidu Server is up and running at "' + OPENVIDU_SERVER_URL + '"')) {
location.assign(OPENVIDU_SERVER_URL + '/accept-certificate');
}
}
}
});
});
}
function createToken(sessionId) { // See https://docs.openvidu.io/en/stable/reference-docs/REST-API/#post-apitokens
return new Promise((resolve, reject) => {
$.ajax({
type: "POST",
url: OPENVIDU_SERVER_URL + "/api/tokens",
data: JSON.stringify({ session: sessionId }),
headers: {
"Authorization": "Basic " + btoa("OPENVIDUAPP:" + OPENVIDU_SERVER_SECRET),
"Content-Type": "application/json"
},
success: response => resolve(response.token),
error: error => reject(error)
});
});
}
What do you mean by this? I am obviously not trying to make it fail. In fact the code works fine if the browser window is active. Did you try my code and follow my steps? It makes it very easy to see the issue.
I’m not waiting for anything. This is just a demo to demonstrate the issue I am having in my app which I’ve explained above several times. Again, my app automatically calls the publish and unpublish methods with some time in between. I’m using setTimeout in this demo to simulate that behavior.
I’m expecting that the publish promise will complete even if the browser window is not active. I’m also expecting that the camera will not turn on after I’ve already called unpublish, just because the browser window was not active. I’m also expecting to not get all those errors, just because the browser window was not active.
Is there something wrong with that? That is how it is done in your openvidu-hello-world tutorial…
We haven’t tested openvidu-browser API with the browser window not active. It seems you found a bug in that situation.
We are focused right now in other priority stuff. We will try to take a look to this issue when we can. If you take a look to the code and found a way to fix it, please let us know.
If you found a workaround to mitigate the issue, please share with the community just in case others find it useful.
Thanks Micael. I understand this is not an issue that people will run into often. But due to the way my app works, it’s more likely for me. My biggest concern is the camera being on when it shouldn’t be, and remaining on even after the session has ended, which will surely upset users. But I’ve found a workaround for that.
I found that it is actually the navigator.mediaDevices.getUserMedia method that doesn’t complete and isn’t cancellable. So I’m not sure what the ultimate solution could be for that.
My workaround is something like this…
let shouldPublish = true;
publisher.on('streamPlaying', (event) => {
if (!shouldPublish) {
event.target.stream.mediaStream.getTracks().forEach((track) => {
track.stop();
});
}
});
shouldPublish gets toggled by my app based on whether the user should be publishing or not. If the window is not active when publish is called, and then unpublish is called while the window is still not active, the publish event will never complete, even when the window is activated. But the streamPlaying event is fired when the window is activated, even if unpublish has been called. So my workaround is to stop all tracks when the streamPlaying event fires, if the user should no longer be publishing.
This, at least turns off the camera, though it does turn on briefly first. There are still many errors in the console, but the most important part is that the camera does not remain on when it shouldn’t be.
Here are some of the errors in case it helps…
This happens when calling unpublish when the window was not active… ERROR:Participant 'con_XGpmhO1Knp' is not streaming media. Code: 105 in Request: method:unpublishVideo params:{} request:undefined
I get many of these when re-activating the window…
Uncaught TypeError: this.stream is undefined
sendIceCandidate Connection.ts:105
onicecandidate WebRtcPeer.ts:64
TypeError: stream is undefined
recvIceCandidate Session.ts:948
In case it helps anyone, my workaround above wasn’t catching all cases (ie. multiple publish/unpublish calls while the browser window is inactive). Here is my new workaround…
const localStreams = [];
let shouldPublish = true;
publisher.on('streamPlaying', (event) => {
localStreams.push(event.target.stream);
if (!shouldPublish) {
cleanupLocalStreams();
}
});
function cleanupLocalStreams() {
while (localStreams.length > 0) {
const stream = localStreams.pop();
const mediaStream = stream.getMediaStream();
try {
stream.disposeMediaStream();
stream.disposeWebRtcPeer();
// this probably isn't needed because `disposeMediaStream` does this.
mediaStream.getTracks().forEach((track) => {
track.stop();
});
} catch(error) {
}
}
}