🐞No Data in Arkime Dashboard

[Wilk4013]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

I installed Clear NDR using docker instructions on 12 core, 32 gig mem, 512 gig mini computer. All seems to be working fine (Kibana, Eve-Box, opensearch dashboards) except there is not data in Arkime Dashboard. Tried recreating container with new pull but still no luck. Can someone help me with this?

Expected Behavior

Expect to see data in Arkime dashboard.

Steps To Reproduce

Install using Docker install instructions from Stamus website.

Docker version

Docker version 27.4.1, build b9d17ea

Docker version

Docker version 27.4.1, build b9d17ea

OS Version

Ubuntu 22.04.5 LTS

Content of the environnement File

N/A

Version of SELKS

version: 1.0.0
arch: linux/amd64
commit: 68aa96734f9eb09db7e90e12eb5cd734d73054e4

Anything else?

No response

[pevma]

In some cases the data does not populate right away , you would either need to wait or make sure you have continuous stream of data/traffic coming in.
Is it the same view if you select All time span in Arkime ?

[Wilk4013]

Peter Yes even with All timespan I still don’t see data in Arkime.

…

On Wed, Dec 25, 2024 at 7:25 AM Peter Manev ***@***.***> wrote: In some cases the data does not populate right away , you would either need to wait or make sure you have continuous stream of data/traffic coming in. Is it the same view if you select All time span in Arkime ? — Reply to this email directly, view it on GitHub <#486 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHE3NOISTZQ3IJKL6SGMEL2HKP4PAVCNFSM6AAAAABUFCKB76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRRHA3DSMZTGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[pevma]

Ok and you have continuous traffic or at least had for some time ?

[Wilk4013]

Yes Peter and I am seeing the constant flow of traffic in eve box and open search.

…

On Thu, Dec 26, 2024 at 3:42 AM Peter Manev ***@***.***> wrote: Ok and you have continuous traffic or at least had for some time ? — Reply to this email directly, view it on GitHub <#486 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHE3NL2DDS6IAXK7QRWHVD2HO6PFAVCNFSM6AAAAABUFCKB76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRSGMZDCNRUGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Wilk4013]

And I have been feeding it data for 5 days On Thu, Dec 26, 2024 at 3:45 AM Charles Wilkerson < ***@***.***> wrote:

…

Yes Peter and I am seeing the constant flow of traffic in eve box and open search. On Thu, Dec 26, 2024 at 3:42 AM Peter Manev ***@***.***> wrote: > Ok and you have continuous traffic or at least had for some time ? > > — > Reply to this email directly, view it on GitHub > <#486 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ACHE3NL2DDS6IAXK7QRWHVD2HO6PFAVCNFSM6AAAAABUFCKB76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRSGMZDCNRUGU> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >

[pevma]

Can you share the output of docker ps -a (example)

docker ps -a
CONTAINER ID   IMAGE                                             COMMAND                  CREATED             STATUS                         PORTS                                                                  NAMES
627bf2ef5ff2   busybox                                           "/bin/sh -c 'chown -…"   About an hour ago   Exited (0) About an hour ago                                                                          config-ownership-fix-UCTNkt
af7ffc98e341   nginx:1.27                                        "/docker-entrypoint.…"   About an hour ago   Up About an hour (healthy)     80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp                          config-nginx-CJDkAt
4fe8df6c1c41   jasonish/suricata:7.0                             "/new_entrypoint.sh"     About an hour ago   Up About an hour                                                                                      config-suricata-wmciTo
8ab834b2c2d1   ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0   "/bin/bash /opt/scir…"   About an hour ago   Up About an hour (healthy)     8000/tcp                                                               config-scirius-VikwzI
86adfe1b028b   config-kibana                                     "./opensearch-dashbo…"   About an hour ago   Up About an hour (healthy)     5601/tcp                                                               config-opensearch-dashboards-HeSGiV
47e3a4c7f7c5   config-arkime                                     "bash -c /start-arki…"   About an hour ago   Up About an hour (healthy)     8005/tcp                                                               config-arkime-dPdnGN
96d87ccc22ce   ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0   "celery -A suricata …"   About an hour ago   Up About an hour (healthy)     8000/tcp                                                               config-celery-beat-SMIYWJ
b1af4b332c9b   ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0   "celery -A suricata …"   About an hour ago   Up About an hour (healthy)     8000/tcp                                                               config-celery-worker-YFknkL
b13e2235c4f3   config-fluentd                                    "tini -- /bin/entryp…"   About an hour ago   Up About an hour               5140/tcp, 24224/tcp                                                    config-fluentd-xfTgbT
a4fe69e0a847   docker:27-cli                                     "docker-entrypoint.s…"   About an hour ago   Up About an hour (healthy)                                                                            config-cron-LPISLd
e51b013765bc   opensearchproject/opensearch:1.3.19               "./opensearch-docker…"   About an hour ago   Up About an hour (healthy)     9200/tcp, 9300/tcp, 9600/tcp, 9650/tcp                                 config-opensearch-gASVZz
521b0b91c640   nginx:1.27                                        "/docker-entrypoint.…"   About an hour ago   Exited (0) About an hour ago                                                                          config-ssl-keygen-SgqAnu
ef863b824eec   rabbitmq:3-management-alpine                      "docker-entrypoint.s…"   About an hour ago   Up About an hour (healthy)     4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 15691-15692/tcp, 25672/tcp   config-rabbitmq-MCXxJJ
08ce80bf8a9c   postgres:17                                       "docker-entrypoint.s…"   About an hour ago   Up About an hour (healthy)     5432/tcp                                                               config-db-tlpYkt
8f844c7da535   jasonish/evebox:master                            "/docker-entrypoint.…"   About an hour ago   Up About an hour                                                                                      config-evebox-cObxnz

also try to see if there is any error message in the Arkime logs

ls -lh config/containers-data/arkime/logs/

[Wilk4013]

Peter I am at work now but I’ll run these commands when get home

…

On Thu, Dec 26, 2024 at 5:21 AM Peter Manev ***@***.***> wrote: Can you share the output of docker ps -a (example) docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 627bf2ef5ff2 busybox "/bin/sh -c 'chown -…" About an hour ago Exited (0) About an hour ago config-ownership-fix-UCTNkt af7ffc98e341 nginx:1.27 "/docker-entrypoint.…" About an hour ago Up About an hour (healthy) 80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp config-nginx-CJDkAt 4fe8df6c1c41 jasonish/suricata:7.0 "/new_entrypoint.sh" About an hour ago Up About an hour config-suricata-wmciTo 8ab834b2c2d1 ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0 "/bin/bash /opt/scir…" About an hour ago Up About an hour (healthy) 8000/tcp config-scirius-VikwzI 86adfe1b028b config-kibana "./opensearch-dashbo…" About an hour ago Up About an hour (healthy) 5601/tcp config-opensearch-dashboards-HeSGiV 47e3a4c7f7c5 config-arkime "bash -c /start-arki…" About an hour ago Up About an hour (healthy) 8005/tcp config-arkime-dPdnGN 96d87ccc22ce ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0 "celery -A suricata …" About an hour ago Up About an hour (healthy) 8000/tcp config-celery-beat-SMIYWJ b1af4b332c9b ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0 "celery -A suricata …" About an hour ago Up About an hour (healthy) 8000/tcp config-celery-worker-YFknkL b13e2235c4f3 config-fluentd "tini -- /bin/entryp…" About an hour ago Up About an hour 5140/tcp, 24224/tcp config-fluentd-xfTgbT a4fe69e0a847 docker:27-cli "docker-entrypoint.s…" About an hour ago Up About an hour (healthy) config-cron-LPISLd e51b013765bc opensearchproject/opensearch:1.3.19 "./opensearch-docker…" About an hour ago Up About an hour (healthy) 9200/tcp, 9300/tcp, 9600/tcp, 9650/tcp config-opensearch-gASVZz 521b0b91c640 nginx:1.27 "/docker-entrypoint.…" About an hour ago Exited (0) About an hour ago config-ssl-keygen-SgqAnu ef863b824eec rabbitmq:3-management-alpine "docker-entrypoint.s…" About an hour ago Up About an hour (healthy) 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 15691-15692/tcp, 25672/tcp config-rabbitmq-MCXxJJ 08ce80bf8a9c postgres:17 "docker-entrypoint.s…" About an hour ago Up About an hour (healthy) 5432/tcp config-db-tlpYkt 8f844c7da535 jasonish/evebox:master "/docker-entrypoint.…" About an hour ago Up About an hour config-evebox-cObxnz also try to see if there is any error message in the Arkime logs ls -lh config/containers-data/arkime/logs/ — Reply to this email directly, view it on GitHub <#486 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHE3NLUENTIK3OUWQR67K32HPKC3AVCNFSM6AAAAABUFCKB76VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRSGQYDQOJQGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Wilk4013]

Peter here is docker ps -a output:

root@clearrva1:/home/charles# docker ps -a
CONTAINER ID   IMAGE                                             COMMAND                  CREATED        STATUS                    PORTS                                                                  NAMES
a751f18508ca   busybox                                           "/bin/sh -c 'chown -…"   43 hours ago   Exited (0) 41 hours ago                                                                          config-ownership-fix-UzsfFH
65ea80ee7618   nginx:1.27                                        "/docker-entrypoint.…"   43 hours ago   Up 41 hours (healthy)     80/tcp, 0.0.0.0:443->443/tcp                                           config-nginx-fSDlxH
0b3468a17be5   jasonish/suricata:7.0                             "/new_entrypoint.sh"     43 hours ago   Up 41 hours                                                                                      config-suricata-mBuHJR
097ff69127db   ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0   "/bin/bash /opt/scir…"   43 hours ago   Up 41 hours (healthy)     8000/tcp                                                               config-scirius-vlcdXK
33d9355aa44f   config-kibana                                     "./opensearch-dashbo…"   43 hours ago   Up 41 hours (healthy)     5601/tcp                                                               config-opensearch-dashboards-dKdpmD
bdc74e2443e5   config-arkime                                     "bash -c /start-arki…"   43 hours ago   Up 41 hours (healthy)     8005/tcp                                                               config-arkime-wqUHSs
83e00cacbea2   ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0   "celery -A suricata …"   43 hours ago   Up 41 hours (healthy)     8000/tcp                                                               config-celery-beat-IatHny
1213eb92cdba   ghcr.io/stamusnetworks/scirius:clear-ndr-v0.9.0   "celery -A suricata …"   43 hours ago   Up 41 hours (healthy)     8000/tcp                                                               config-celery-worker-cCPyjR
0cc1825aa725   postgres:17                                       "docker-entrypoint.s…"   43 hours ago   Up 41 hours (healthy)     5432/tcp                                                               config-db-fKXprO
b75d2911a29b   nginx:1.27                                        "/docker-entrypoint.…"   43 hours ago   Exited (0) 41 hours ago                                                                          config-ssl-keygen-fKlPjL
4f4182fe6025   config-fluentd                                    "tini -- /bin/entryp…"   43 hours ago   Up 41 hours               5140/tcp, 24224/tcp                                                    config-fluentd-Zxvany
fae51ea5239b   rabbitmq:3-management-alpine                      "docker-entrypoint.s…"   43 hours ago   Up 41 hours (healthy)     4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 15691-15692/tcp, 25672/tcp   config-rabbitmq-yKiJEN
9ec9d965f325   opensearchproject/opensearch:1.3.19               "./opensearch-docker…"   43 hours ago   Up 41 hours (healthy)     9200/tcp, 9300/tcp, 9600/tcp, 9650/tcp                                 config-opensearch-WfxtKw
bebf6921a55d   docker:27-cli                                     "docker-entrypoint.s…"   43 hours ago   Up 41 hours (healthy)                                                                            config-cron-nGHfVq
dc04f31b502b   jasonish/evebox:master                            "/docker-entrypoint.…"   43 hours ago   Up 41 hours                                                                                      config-evebox-dxtNPR
8f9aae9986b6   ghcr.io/stamusnetworks/arkimeviewer:master        "bash -c /start-arki…"   2 days ago     Up 41 hours (healthy)     0.0.0.0:8005->8005/tcp                                                 arkime
e23018aca34a   elastic/logstash:7.17.20                          "/usr/local/bin/dock…"   2 days ago     Up 41 hours (healthy)     5044/tcp, 9600/tcp                                                     logstash
bfc754ca664e   nginx                                             "/docker-entrypoint.…"   2 days ago     Created                                                                                          nginx
a5554bf4ba83   ghcr.io/stamusnetworks/scirius:selks              "/bin/bash /opt/scir…"   2 days ago     Up 41 hours (healthy)     8000/tcp                                                               scirius
1b7b78ddb66f   elastic/kibana:7.17.20                            "/bin/tini -- /usr/l…"   2 days ago     Up 41 hours (healthy)     5601/tcp                                                               kibana
d41d013db325   docker:latest                                     "dockerd-entrypoint.…"   2 days ago     Up 41 hours (healthy)     2375-2376/tcp                                                          cron
57781cb7e70b   elastic/elasticsearch:7.17.20                     "/bin/tini -- /usr/l…"   2 days ago     Up 41 hours (healthy)     9200/tcp, 9300/tcp                                                     elasticsearch
9052b46c3c47   postgres                                          "docker-entrypoint.s…"   2 days ago     Up 41 hours (healthy)     5432/tcp                                                               selks-db-1
5bb203ed769c   jasonish/suricata:master-amd64                    "/etc/suricata/new_e…"   2 days ago     Up 41 hours                                                                                      suricata
25792a60d6f8   jasonish/evebox:master                            "/docker-entrypoint.…"   2 days ago     Up 41 hours                                                                                      evebox
1b776d53a4e6   busybox                                           "ls /sys/class/net"      2 days ago     Exited (0) 41 hours ago                                                                          agitated_dijkstra
d6c2d56c7c45   portainer/portainer-ce:latest                     "/portainer"             2 days ago     Up 43 hours               8000/tcp, 9443/tcp, 0.0.0.0:9000->9000/tcp                             pedantic_lumiere
root@clearrva1:/home/charles# 

[pevma]

Thanks.

Can you try upgrading please : https://docs.clearndr.io/docs/start/common-operations#upgrading
Also what is the output of:

 tail -30  config/containers-data/arkime/logs/capture.log 

[Wilk4013]

[root@lrva1-summernet-net charles]#  tail -30  config/containers-data/arkime/logs/capture.log 
Dec 23 11:04:24 http.c:406 arkime_http_curlm_check_multi_info(): 2/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 793/160 0ms 5ms
Dec 23 11:04:24 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 201 http://elasticsearch:9200/arkime_dstats/_doc/fdfd917cb55a-772-5 793/161 0ms 5ms
Dec 23 11:04:26 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 793/161 0ms 6ms
Dec 23 11:04:28 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 793/161 0ms 7ms
Dec 23 11:04:29 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 201 http://elasticsearch:9200/arkime_dstats/_doc/fdfd917cb55a-773-5 793/161 0ms 6ms
Dec 23 11:04:30 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 793/161 0ms 6ms
Dec 23 11:04:32 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 793/161 0ms 7ms
Dec 23 11:04:34 http.c:406 arkime_http_curlm_check_multi_info(): 2/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 793/161 0ms 6ms
Dec 23 11:04:34 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 201 http://elasticsearch:9200/arkime_dstats/_doc/fdfd917cb55a-774-5 793/161 0ms 6ms
Dec 23 11:04:34 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/_cat/health?format=json 0/209 0ms 1ms
Dec 23 11:04:36 http.c:406 arkime_http_curlm_check_multi_info(): 1/3 ASYNC 200 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 793/161 0ms 7ms
Dec 23 11:04:38 http.c:656 arkime_http_curl_close_callback(): Couldn't connect http://elasticsearch:9200 (11, 21493) 
Dec 23 11:04:38 http.c:406 arkime_http_curlm_check_multi_info(): 1/0 ASYNC 0 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 0/0 0ms 26ms
Dec 23 11:04:38 http.c:656 arkime_http_curl_close_callback(): Couldn't connect http://elasticsearch:9200 (11, 21498) 
Dec 23 11:04:38 http.c:406 arkime_http_curlm_check_multi_info(): 1/0 ASYNC 0 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 0/0 0ms 0ms
Dec 23 11:04:38 http.c:439 arkime_http_curlm_check_multi_info(): Response length=0 :>

Dec 23 11:04:39 http.c:656 arkime_http_curl_close_callback(): Couldn't connect http://elasticsearch:9200 (11, 21503) 
Dec 23 11:04:39 http.c:406 arkime_http_curlm_check_multi_info(): 1/0 ASYNC 0 http://elasticsearch:9200/arkime_dstats/_doc/fdfd917cb55a-775-5 0/0 0ms 0ms
Dec 23 11:04:39 http.c:656 arkime_http_curl_close_callback(): Couldn't connect http://elasticsearch:9200 (11, 21508) 
Dec 23 11:04:39 http.c:406 arkime_http_curlm_check_multi_info(): 1/0 ASYNC 0 http://elasticsearch:9200/arkime_dstats/_doc/fdfd917cb55a-775-5 0/0 0ms 0ms
Dec 23 11:04:39 http.c:439 arkime_http_curlm_check_multi_info(): Response length=0 :>

Dec 23 11:06:19 main.c:255 parse_args(): WARNING: gethostname doesn't return a fully qualified name and getdomainname failed, this may cause issues when viewing pcaps, use the --host option - fdfd917cb55a
Dec 23 11:06:19 config.c:281 arkime_config_int(): INFO: Reseting dbBulkSize since 300000 is less then the min 500000
Dec 23 11:06:19 http.c:318 arkime_http_send_sync(): 1/1 SYNC 200 http://elasticsearch:9200/_template/arkime_sessions3_template?filter_path=**._meta 0/96 6ms 51ms
Dec 23 11:06:20 http.c:318 arkime_http_send_sync(): 1/1 SYNC 503 http://elasticsearch:9200/arkime_sequence/_doc/fn-fdfd917cb55a 0/671 0ms 187ms
Dec 23 11:06:20 http.c:318 arkime_http_send_sync(): 1/1 SYNC 503 http://elasticsearch:9200/arkime_stats/_doc/fdfd917cb55a 0/313 0ms 20ms
Dec 23 11:06:20 http.c:318 arkime_http_send_sync(): 1/1 SYNC 503 http://elasticsearch:9200/arkime_fields/_search?size=3000 0/161 0ms 230ms
Dec 23 11:06:20 db.c:2453 arkime_db_load_fields(): ERROR - Couldn't download arkime_fields, database (http://elasticsearch:9200) might be down or not initialized

[Wilk4013]

Peter I ran the update and I am still getting "Oh no, Arkime is empty! There is no data to search"

[pevma]

What is the output of those two commands:

ls -lh  config/containers-data/suricata/logs/fpc/
ls -lh  config/fpc/ 

[Wilk4013]

root@clearrva1:/home/charles# ls -lh config/containers-data/suricata/logs/fpc/
total 0