There was an error on my Kibana dashboard, what should I do?

Issues with deployment
1

I am currently trying OpenVidu Pro in an aws environment using cloudformation with a 15-day free trial.
Since 2.15.0 was recently released, I attempted to upgrade from 2.14.0.
The OpenVidu server is working fine, and I can make video calls.

However, accessing OpenVidu Sessions (/kibana/app/kibana#/dashboard/2f3fd320-4964-11e9-9bb9-8ff7364e31cb) on the kibana dashboard screen resulted in the attached error.

スクリーンショット 2020-07-13 14.23.56
スクリーンショット 2020-07-13 14.23.563358×3356 1.09 MB

I checked the response from elasticsearch in chrome’s developper tools and found

/kibana/internal/search/es

In response to the request to

{“statusCode”:400, “error”: “Bad Request”," message":"[query_shard_exception] failed to create query: For input string: For input string: 2020-07-12T15: 00:00.000Z"", “attributes”:{“error”:{“type “: “query_shard_exception”, “reason”: “failed to create query: For input string: For input string: 2020-07-12T 15:00:00.000Z””, “index_uuid”: “rKlk1m8 tSIKEuIdw0pO1PQ”, “index”: “openvidu”, “caused _by”:{“type”: “number_format_exception”,” reason": "For input string: "For input string: “2020-07-12T15:00: 00.000Z\” }}}}%

I seemed to be getting an error response of I’m trying it out by setting the query parameters for the timestamp field to a string of dates to unix timestamp, and then made a curl request for The response was returned successfully.

Is there any way to prevent this error from occurring in the dashboard?

2

Hi, there was a problem in version 2.15.0 with ELK. Try to patch it to version 2.15.1. You have instructions here

3

Same error for us here. 2.14.0 updated to 2.15.0 produced this error for us. 2.15.0 to 2.15.1 update and error still present. Kibana unusable at this point.

Suggestions?

1 Like
4

We get this error on the following dashboards:

OpenVidu Recordings
OpenVidu Sessions

OpenVidu Clustering works without any errors.

1 Like
5

Hello. Thank you for your reply.

I was able to upgrade to 2.15.1 using the method you gave me.

ubuntu@ip-172-31-17-56:~$ sudo docker ps
CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                              NAMES
119bc205bf7d        openvidu/openvidu-server-pro:2.15.1                   "/bin/bash -c 'expor…"   10 minutes ago      Up 10 minutes                                          openvidu_openvidu-server_1
bdb4d168e4ac        docker.elastic.co/elasticsearch/elasticsearch:7.8.0   "/tini -- /usr/local…"   10 minutes ago      Up 10 minutes       0.0.0.0:9200->9200/tcp, 9300/tcp   openvidu_elasticsearch_1
3ac61e905a16        openvidu/openvidu-coturn:1.0.0                        "/bin/sh -c /usr/loc…"   10 minutes ago      Up 10 minutes                                          openvidu_coturn_1
24b1848a5610        docker.elastic.co/beats/filebeat:7.8.0                "/usr/local/bin/dock…"   10 minutes ago      Up 10 minutes                                          openvidu_filebeat_1
233175df08b4        openvidu/openvidu-proxy:3.0.0                         "/bin/sh -c 'htpassw…"   10 minutes ago      Up 10 minutes                                          openvidu_nginx_1
1d2a6cf03eb1        docker.elastic.co/kibana/kibana:7.8.0                 "/usr/local/bin/dumb…"   10 minutes ago      Up 10 minutes       0.0.0.0:5601->5601/tcp             openvidu_kibana_1
0ac3ab1225fb        openvidu/openvidu-redis:1.0.0                         "docker-entrypoint.s…"   10 minutes ago      Up 10 minutes                                          openvidu_redis_1

After restarting the OpenVidu server, I went to Kibana’s OpenVidu Sessions’ I looked at the dashboard. However, the same error as yesterday was occurring.

I checked the network tab of the developer tools in chrome and found that the error occurred with the following request.

request:
POST /kibana/internal/search/es
{"params":{"ignoreThrottled":true,"preference":1594687260739,"index":"openvidu","body":{"aggs":{"1":{"avg":{"field":"msSinceEndpointCreation"}}},"size":0,"stored_fields":["*"],"script_fields":{"sizeFormatted":{"script":{"source":"if (doc.containsKey('size')) {\n  if (doc['size'].size()==0) {\n    return '';\n  } else {\n    return doc['size'].value;\n  }\n} else {\n  return '';\n}","lang":"painless"}}},"docvalue_fields":[{"field":"createdAt","format":"date_time"},{"field":"destroyedAt","format":"date_time"},{"field":"startTime","format":"date_time"},{"field":"timestamp","format":"date_time"}],"_source":{"excludes":[]},"query":{"bool":{"must":[{"match_all":{}}],"filter":[{"match_phrase":{"elastic_type.keyword":{"query":"kms"}}},{"match_phrase":{"type.keyword":{"query":"MediaFlowOutStateChange"}}},{"match_phrase":{"state.keyword":{"query":"FLOWING"}}},{"range":{"timestamp":{"gte":"2020-07-13T15:00:00.000Z","lte":"2020-07-14T14:59:59.999Z","format":"strict_date_optional_time"}}}],"should":[],"must_not":[]}}},"rest_total_hits_as_int":true,"ignore_unavailable":true,"ignore_throttled":true,"timeout":"30000ms"},"serverStrategy":"es"}

response:
{
  "statusCode": 400,
  "error": "Bad Request",
  "message": "[query_shard_exception] failed to create query: For input string: \"2020-07-13T15:00:00.000Z\"",
  "attributes": {
    "error": {
      "type": "query_shard_exception",
      "reason": "failed to create query: For input string: \"2020-07-13T15:00:00.000Z\"",
      "index_uuid": "rKlk1m8tSIKEuIdw0pO1PQ",
      "index": "openvidu",
      "caused_by": {
        "type": "number_format_exception",
        "reason": "For input string: \"2020-07-13T15:00:00.000Z\""
      }
    }
  }
}

This error response is the same as yesterday.

6

Hi @caffelattenonsugar @jcdmacleod:

Unfortunately, your deployment was affected by the bug, and openvidu index was removed from elasticsearch. Data can’t be recovered, but you can fix the stack by doing the following:
1). Stop OpenVidu:
In /opt/openvidu

./openvidu stop

2). Remove corrupted previous data:

rm -rf /opt/openvidu/elasticsearch

3). Recreate elasticsearch directory:

mkdir /opt/openvidu/elasticsearch && chown 1000:1000 /opt/openvidu/elasticsearch

4). Start OpenVidu:
In /opt/openvidu

./openvidu start
7

Hi @cruizba

I ran the command you gave me and it works fine. Thank you!

1 Like