Voice detection layout

OpenVidu v2 Issues developing apps
1

Hello,

I was implementing a voice detection layout which try to do bigger the player from the people who is speaking (using these events: https://openvidu.io/docs/advanced-features/speech-detection/). I got enlarge the player for the first speaker but is not working for the second one. I am logging in the console browser and looks like not receive the events for the second user.

Is this normal os am I doing something wrong? Any advise?

Thanks in advance,

Beka

2

Hi,

As docs says (https://openvidu.io/api/openvidu-browser/classes/publisherspeakingevent.html) speach events are only triggered for remote streams. right now.
We can study the possiblity of also supporting them in Publishers objects.

3

mmmmm, yeps, I understand that, but you say in this part of the doc ( https://openvidu.io/docs/advanced-features/speech-detection/) : “with these events it is really easy to build a layout that can make the main speaker video the bigger one, and alternate the main view between the participants of a session as they take the floor.” I am trying to figure out why is working for the first participant and not for the second one. I mean, the second user is the remote one for the first user and his/her events are not received by any of them.

4

All remote Streams (those owned by a Subscriber object) should trigger publisherStartSpeaking and publisherStopSpeaking. Those Streams must of course have audio tracks (property hasAudio to true).

If you say that in a 1:1 session one user is receiving these events for the other peer, but the other user doesn’t, then there must be something wrong with the event subscription for that specific user.

5

I am subscribing to the publisherSpeaking events directly in the joinToSession function in the video-room.component.ts. Supposedly, the subscription apply to any user who was joining the event. I am not really sure about how to debug it. Any advise to debbug it is welcome. Thanks.

joinToSession() {
    const optionsConf = {
      publisherSpeakingEventsOptions: {
        interval: 50,   // Frequency of the polling of audio streams in ms
        threshold: -50  // Threshold volume in dB
    }
    };

    this.OV = new OpenVidu();
    this.OV.setAdvancedConfiguration(optionsConf);
    this.OVScreen = new OpenVidu();
    this.OVScreen.setAdvancedConfiguration(optionsConf);
    this.session = this.OV.initSession();
    this.sessionScreen = this.OVScreen.initSession();
    this.subscribeToUserChanged();
    this.subscribeToStreamCreated();
    this.subscribedToStreamDestroyed();
    this.subscribedToChat();
    this.publisherStartSpeakingSubs();
    this.publisherStopSpeakingSubs();
    this.subscribeToEnlargeElementSignal();
    this.privateChatMessagesSignals();
    this.requestPermissionSignal();
    this.sessionDisconnectedSubs();
    this.connectionDestroyedSubs();
    this.connectToSession();
    this.updateComponent();
  }
6

Hey, how did you get to work in the first time. I am having doubts about how to configure that. I need to send the speaker to the main video (bigger then other). Something like google meet.

7

So whats your issues?
Thanks
Vipin

8

Hi @vipin_mishra no issues, I was just wondering how to implement the right way.

I already got to the session.on(‘publicherStartSpeaking’, () => {}), but got confuse how I get this information and set the video from this publisher as a main video for all the participants in the call.

9

are you using angular or any other frameworks

10

I am using React! I am using voice recognition, changing the state getting the video from subscribers and putting in the mainVideo, but the problem it is only working for one user.

11

In angular we layout components using that it’s easily done for Reactjs I think you have to made.
And publisher speaking event does not work on publisher side.
So you have to use voice or volume detection web api to change on publisher side.
Thanks
Vipin

12

Hi @vipin_mishra, that was what I needed. Do you know what is the best voice detection api to use? I am looking into Web Speech API, but only works in Chrome.

13

Check This link OpenVidu Docs . Its worked you have to decide when volume got high at any point then you know pulisher started speaking

14

Hey, I tried that and the problem is I am getting a lot of background noise, and it is acting all the time. Do you have an example about how you did implement this?

15

I developed for a client but I don’t remember I have code or not available at this time.

Let me check it
Thanks
Vipin