# Image analysis (Object detection) on ongoing streaming video captured through ip camera

**URL:** <https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376>\
**Category:** How to implement?\
**Tags:** v2\
**Created:** [September 15, 2021, 10:24am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376 "2021-09-15T10:24:06Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 15, 2021, 10:24am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/1 "2021-09-15T10:24:06Z")

</div>

Hello @cruizba  
I have one more question: I need to do image analysis on ongoing streaming video which is captured through ip camera.  
**usually I have to perform object detection**  
Is any documnentation there ?. I saw [OpenVidu Docs](https://docs.openvidu.io/en/2.19.0/advanced-features/filters/) but this is what I am not looking for.  
If I have to add/perform openCV or Deeplearnig algorithm which part of the code should I modify is it on the browser side (openvidu-browser-2.19.0.js) or any hint regarding this would be appreciated.

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 16, 2021, 9:32am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/2 "2021-09-16T09:32:07Z")

</div>

@cruizba @CSantosM  
May I request an hint for the above mentioned question ?

---

<div class="post-metadata">

**Author:** ![micael.gallego](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/micael.gallego/32/2_2.png) [@micael.gallego](https://openvidu.discourse.group/u/micael.gallego)\
**Post date:** [September 16, 2021, 9:53am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/3 "2021-09-16T09:53:45Z")

</div>

If you want to perform object detection in the server you have to implement a custom Kurento Module:

[https://doc-kurento.readthedocs.io/en/stable/user/writing\_modules.html](https://doc-kurento.readthedocs.io/en/stable/user/writing_modules.html)

Then you have to create a custom Kurento Docker image with the module installed:

[https://doc-kurento.readthedocs.io/en/stable/user/writing\_modules.html#installing-in-docker](https://doc-kurento.readthedocs.io/en/stable/user/writing_modules.html#installing-in-docker)

Finally you have to configure the session to use the module created:

[https://docs.openvidu.io/en/2.19.0/advanced-features/filters/](https://docs.openvidu.io/en/2.19.0/advanced-features/filters/)

Hope it helps.

Regards

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 17, 2021, 3:24pm UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/7 "2021-09-17T15:24:53Z")

</div>

I had created a sample kurento custom filter.  
If I had to call the filter in ip camera running tutorial which part of the code should I tweak  
As I see in link [OpenVidu Docs](https://docs.openvidu.io/en/2.19.0/advanced-features/filters/)  
There is a snippet code

```auto
ConnectionProperties connectionProperties = new ConnectionProperties.Builder()
    .type(ConnectionType.WEBRTC)
    .data("user_data")
    .role(OpenViduRole.PUBLISHER)
    .kurentoOptions(
        new KurentoOptions.Builder()
            .allowedFilters(new String[]{"GStreamerFilter", "FaceOverlayFilter"})
            .build())
    .build();
Connection connection = session.createConnection(connectionProperties);
String token = connection.getToken();

```

So I call the filter name over here:  
`allowedFilters(new String[]{"GStreamerFilter", "FaceOverlayFilter"})`

or how this gone work any hint or support would be appreciated  
@micael.gallego @cruizba @CSantosM

---

<div class="post-metadata">

**Author:** ![micael.gallego](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/micael.gallego/32/2_2.png) [@micael.gallego](https://openvidu.discourse.group/u/micael.gallego)\
**Post date:** [September 17, 2021, 4:53pm UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/8 "2021-09-17T16:53:09Z")

</div>

You have to allow your own filter in “allowedFilters”.

Also, to connect a IP\_CAMERA, the type should be type(ConnectionType.IPCAM)

Regards

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 21, 2021, 11:45am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/9 "2021-09-21T11:45:09Z")

</div>

Hello @micael.gallego  
I am not able to run docker build inside the docker container kms  
is it I need to install docker inside the container ??  
or in this link [Writing Kurento Modules — Kurento 6.16.0 documentation](https://doc-kurento.readthedocs.io/en/stable/user/writing_modules.html#installing-in-docker) as said it’s not working  
May I know what is the possible reason ??  
Any hint is appreciated

Regards,

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 21, 2021, 12:53pm UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/10 "2021-09-21T12:53:34Z")

</div>

Hello @micael.gallego @cruizba @CSantosM  
As said in above thread by @micael.gallego I have tweaked the lines from 70 in **MyRestController.java**  
I don’t get that in place of type(ConnectionType.WEBRTC) I used type(ConnectionType.IPCAM)  
and in allowedFilters(new String[]{“opencvsample”}) I had given my custom module filter

```auto
// Create a Connection for the client
		ConnectionProperties connectionProperties = new ConnectionProperties.Builder()
				.type(ConnectionType.IPCAM)
				.data("user_data")
				.role(OpenViduRole.SUBSCRIBER)
				.kurentoOptions(
					new KurentoOptions.Builder()
						.allowedFilters(new String[]{"opencvsample"})
						.build())
				.build();
		String token = null;
		try {
			token = this.session.createConnection(connectionProperties).getToken();
		} catch (OpenViduHttpException e)

```

When I run the application I get the error as **Error creating Connection for session MySurveillanceSession: 400**  
But when I keep the type(ConnectionType.WEBRTC) I don’t see any errors and other hand I can’t see filter applied to video(captured from ip camera)

---

<div class="post-metadata">

**Author:** ![micael.gallego](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/micael.gallego/32/2_2.png) [@micael.gallego](https://openvidu.discourse.group/u/micael.gallego)\
**Post date:** [September 21, 2021, 5:47pm UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/11 "2021-09-21T17:47:19Z")

</div>

You don’t have to use docker build inside the container. You have to use docker build in your host to create a new kms image with your plugin.

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 24, 2021, 2:38pm UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/14 "2021-09-24T14:38:16Z")

</div>

Hello @micael.gallego @CSantosM @cruizba  
this is filter that I had in kms used by openvidu

```auto
root@df5658176b0b:/# kurento-media-server -v
Kurento Media Server version: 6.16.0
Found modules:
        'chroma' version 6.16.0
        'core' version 6.16.0
        'elements' version 6.16.0
        'filters' version 6.16.0
        'opencvsample' version 0.0.1~0.g8fe867d
        'opencvtest' version 0.0.1~0.g5732698

```

Is this defined correct in openvidu-ipcamera tutorial ??  
I had tweaked the MyRestController.Java file as shown below

```auto
// Create a Connection for the client
		ConnectionProperties connectionProperties = new ConnectionProperties.Builder()
				.type(ConnectionType.IPCAM)
				.role(OpenViduRole.SUBSCRIBER)
				.kurentoOptions(
					new KurentoOptions.Builder()
						.allowedFilters(new String[]{"opencvtest"})
						.build())
				.build();
		String token = null;
		try {
			token = this.session.createConnection(connectionProperties).getToken();
		} catch (OpenViduHttpException e) {
			if (e.getStatus() == 404) {
				// Session was closed in openvidu-server. Re-create it
				createOpenViduSession();
				publishCameras();
				token = this.session.createConnection(connectionProperties).getToken();
			} else {
				return generateError(model,
						"Error creating Connection for session " + SESSION_ID + ": " + e.getMessage());

```

And when I run this I get the following error:  
**Error creating Connection for session MySurveillanceSession: 400**

---

<div class="post-metadata">

**Author:** ![cruizba](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/cruizba/32/18_2.png) [@cruizba](https://openvidu.discourse.group/u/cruizba)\
**Post date:** [September 24, 2021, 7:20pm UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/15 "2021-09-24T19:20:02Z")

</div>

Hi @Hrishkesh_Pattepu .

IP cameras are publishers, not subscribers. I think this is what is wrong in your code:

```auto
ConnectionProperties connectionProperties = new ConnectionProperties.Builder()
				.type(ConnectionType.IPCAM)
				.role(OpenViduRole.SUBSCRIBER) <------ DELETE THIS
				.kurentoOptions(
					new KurentoOptions.Builder()
						.allowedFilters(new String[]{"opencvtest"})
						.build())
				.build();

```

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 27, 2021, 7:27am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/16 "2021-09-27T07:27:49Z")

</div>

Hello @cruizba  
I had tried the way you said, I deleted

```auto
ConnectionProperties connectionProperties = new ConnectionProperties.Builder()
				.type(ConnectionType.IPCAM)
				.role(OpenViduRole.PUBLISHER) <------ Replaced with publisher
				.kurentoOptions(
					new KurentoOptions.Builder()
						.allowedFilters(new String[]{"opencvtest"})
						.build())
				.build();

```

I get the following error  
**Error creating Connection for session MySurveillanceSession: 400**  
I know how to create custom module at lower level of kurento.  
But not getting to know how to add my plugin at openvidu level  
May I request for an help as said I’am new to this platform ??

Regards,

---

<div class="post-metadata">

**Author:** ![pabloFuente](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/pablofuente/32/2536_2.png) [@pabloFuente](https://openvidu.discourse.group/u/pabloFuente)\
**Post date:** [September 27, 2021, 8:02am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/17 "2021-09-27T08:02:17Z")

</div>

@Hrishkesh_Pattepu Let me do a couple of tests with some filters applied to IPCAMs, and I’ll come back with some more information.

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 27, 2021, 8:03am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/18 "2021-09-27T08:03:56Z")

</div>

Sure @pabloFuente, thank you so much!

---

<div class="post-metadata">

**Author:** ![pabloFuente](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/pablofuente/32/2536_2.png) [@pabloFuente](https://openvidu.discourse.group/u/pabloFuente)\
**Post date:** [September 27, 2021, 9:40am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/19 "2021-09-27T09:40:22Z")

</div>

I can confirm everything works just as expected when applying real-time filters over IPCAM streams. The only limitation is the following one:

You must control the application of the filter over the IPCAM stream through a MODERATOR connection. Audio/Video filters are managed via openvidu-browser.js library, just like this: [OpenVidu Docs](https://docs.openvidu.io/en/2.20.0/advanced-features/filters/#2b-or-apply-the-filter-dynamically-after-publishing-the-stream-whenever-you-want)

But as IP cameras are Connections initialized through REST API ([OpenVidu Docs](https://docs.openvidu.io/en/2.20.0/advanced-features/ip-cameras/#how-to-publish-ip-cameras)), then the only way to apply and remove filters over its stream is through a different Connection (a regular one, of type WEBRTC), which will need role MODERATOR to do so over. Connections with role MODERATOR are the only ones able to apply filters over other Connection’s streams. To sum up:

- Publish the IP camera stream through REST API just like this: [OpenVidu Docs](https://docs.openvidu.io/en/2.20.0/advanced-features/ip-cameras/#how-to-publish-ip-cameras), making sure you provide the **`kurentoOptions.allowedFilters`** parameter.
- Using openvidu-browser, and thanks to a Connection initialized with **`MODERATOR`** role, apply a video filter over the Stream object of the IP Camera participant (use this method: [Stream | OpenVidu Browser - v2.19.0](https://docs.openvidu.io/en/2.20.0/api/openvidu-browser/classes/stream.html#applyfilter)).

You should start receiving the IP Camera feed with the filter applied just after calling the previous openvidu-browser.js method.

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 27, 2021, 12:59pm UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/21 "2021-09-27T12:59:26Z")

</div>

Hello @pabloFuente  
Thank you for the information provided  
May I request for the example.  
In your case if you had tested something can you share it ??  
The points mentioned above looks confusing to me.

May I know what this phrase means ??  
It would be better, If get some practical example on the below lines.  
like what to add or which part of the code should I tweak ??

> [@pabloFuente](#):
>
> But as IP cameras are Connections initialized through REST API ([OpenVidu Docs](https://docs.openvidu.io/en/2.20.0/advanced-features/ip-cameras/#how-to-publish-ip-cameras)), then the only way to apply and remove filters over its stream is through a different Connection (a regular one, of type WEBRTC), which will need role MODERATOR to do so over. Connections with role MODERATOR are the only ones able to apply filters over other Connection’s streams. To sum up:
> 
> - Publish the IP camera stream through REST API just like this: [OpenVidu Docs](https://docs.openvidu.io/en/2.20.0/advanced-features/ip-cameras/#how-to-publish-ip-cameras), making sure you provide the **`kurentoOptions.allowedFilters`** parameter.
> - Using openvidu-browser, and thanks to a Connection initialized with **`MODERATOR`** role, apply a video filter over the Stream object of the IP Camera participant (use this method: [Stream | OpenVidu Browser - v2.19.0](https://docs.openvidu.io/en/2.20.0/api/openvidu-browser/classes/stream.html#applyfilter)).

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 27, 2021, 2:51pm UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/22 "2021-09-27T14:51:19Z")

</div>

Hello @pabloFuente @micael.gallego @cruizba  
May I request you to address the above question please ??

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 28, 2021, 6:59am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/23 "2021-09-28T06:59:38Z")

</div>

Hello @pabloFuente  
May I request for example, where I can understand it much better.  
It would be much helpful to me.

Regards,

---

<div class="post-metadata">

**Author:** ![micael.gallego](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/micael.gallego/32/2_2.png) [@micael.gallego](https://openvidu.discourse.group/u/micael.gallego)\
**Post date:** [September 28, 2021, 8:30am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/24 "2021-09-28T08:30:51Z")

</div>

We are going to publish a sample project soon.

---

<div class="post-metadata">

**Author:** ![pabloFuente](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/pablofuente/32/2536_2.png) [@pabloFuente](https://openvidu.discourse.group/u/pabloFuente)\
**Post date:** [September 28, 2021, 9:54am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/26 "2021-09-28T09:54:41Z")

</div>

Here you go: [GitHub - pabloFuente/openvidu-filter-ipcam](https://github.com/pabloFuente/openvidu-filter-ipcam)  
That is a simple example based on tutorial [openvidu-tutorials/openvidu-filters at master · OpenVidu/openvidu-tutorials · GitHub](https://github.com/OpenVidu/openvidu-tutorials/tree/master/openvidu-filters)

It allows you to publish an IP camera and apply filters to it. See the README file for isntructions to run it.

---

<div class="post-metadata">

**Author:** ![Hrishkesh\_Pattepu](https://yyz2.discourse-cdn.com/free1/user_avatar/openvidu.discourse.group/hrishkesh_pattepu/32/1635_2.png) [@Hrishkesh\_Pattepu](https://openvidu.discourse.group/u/Hrishkesh_Pattepu)\
**Post date:** [September 28, 2021, 10:06am UTC](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376/27 "2021-09-28T10:06:55Z")

</div>

Thank you so much @pabloFuente!!

[Next page](https://openvidu.discourse.group/t/image-analysis-object-detection-on-ongoing-streaming-video-captured-through-ip-camera/3376.md?page=2)