alpha breakdown-shortest search

[EnglishVillage]

What version of Dgraph are you using?

Dgraph version : v1.2.1
Dgraph SHA-256 : 3f18ff84570b2944f4d75f6f508d55d902715c7ca2310799cc2991064eb046f8
Commit SHA-1 : ddcda92
Commit timestamp : 2020-02-06 15:31:05 -0800
Branch : HEAD
Go version : go1.13.5

Have you tried reproducing the issue with the latest release?

yes

What is the hardware spec (RAM, OS)?

uname -a:
Linux test102 3.10.0-862.14.4.el7.x86_64 #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
16g

Steps to reproduce the issue (command/config used to run Dgraph).

dgraph zero -w /dgraph/dgraphdata/zero/zw
(The server is a test machine, so alpha setup memory is small)
dgraph alpha --lru_mb 1024 --zero 127.0.0.1:5080 -o 100
dgraph-ratel -port 8001

Expected behaviour and actual result.

right:
in dgraph-ratel:
{
path as shortest( from: 0x3bf7fd, to: 0x3e4577, depth: 1, numpaths: 1) {
invest_capital.base_company,~invest_capital.base_company
}
all(func: uid(path)) {
uid,xid,base_company.name,invest_capital.name_short,base_company.parent_company_group_name
}
}

right result:
in dgraph-ratel:
{
"data": {
"all": []
},
"extensions": {
"server_latency": {
"parsing_ns": 129898,
"processing_ns": 4531001,
"encoding_ns": 2745,
"assign_timestamp_ns": 682553,
"total_ns": 5447407
},
"txn": {
"start_ts": 26300
},
"metrics": {
"num_uids": {
"": 1,
"base_company.name": 0,
"base_company.parent_company_group_name": 0,
"invest_capital.base_company": 5,
"invest_capital.name_short": 0,
"uid": 0,
"xid": 0,
"~invest_capital.base_company": 5
}
}
}
}

========================================================

error1(in dgraph-ratel):
{
path as shortest( from: 0x3bf7fd, to: 0x3e4577, depth: 1, numpaths: 2) {
invest_capital.base_company,~invest_capital.base_company
}
all(func: uid(path)) {
uid,xid,base_company.name,invest_capital.name_short,base_company.parent_company_group_name
}
}

error msg(no alpha error msg,no zero error msg):
Error Name: TypeError
Message: Failed to fetch
URL: undefined

---

error2(in pydgraph):
{
path as shortest( from: 0x3bf7fd, to: 0x3e4577, depth: 1, numpaths: 2) {
invest_capital.base_company,~invest_capital.base_company
}
all(func: uid(path)) {
uid,xid,base_company.name,invest_capital.name_short,base_company.parent_company_group_name
}
}

error msg:
ERROR in app: Exception on /python/api/dgraph/get/shortest [POST]
Traceback (most recent call last):
File "/usr/local/python3.5.2/lib/python3.5/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/python3.5.2/lib/python3.5/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/python3.5.2/lib/python3.5/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/python3.5.2/lib/python3.5/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/local/python3.5.2/lib/python3.5/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/python3.5.2/lib/python3.5/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "DgraphImpl.py", line 83, in get_shortest
"result": PydgraphUtils.get_shortest(uid1, uid2, find_fields, show_fields, depth=depth, numpaths=numpaths)}
File "/home/esuser/NLP35/utils/PydgraphUtils.py", line 806, in get_shortest
d = __get_query(query, best_effort=best_effort)
File "/home/esuser/NLP35/utils/PydgraphUtils.py", line 419, in __get_query
res = txn.query(query, variables=variables)
File "/usr/local/python3.5.2/lib/python3.5/site-packages/pydgraph/txn.py", line 62, in query
return self.do_request(req, timeout=timeout, metadata=metadata, credentials=credentials)
File "/usr/local/python3.5.2/lib/python3.5/site-packages/pydgraph/txn.py", line 110, in do_request
self._common_except_mutate(query_error)
File "/usr/local/python3.5.2/lib/python3.5/site-packages/pydgraph/txn.py", line 160, in _common_except_mutate
raise error
File "/usr/local/python3.5.2/lib/python3.5/site-packages/pydgraph/txn.py", line 88, in do_request
credentials=credentials)
File "/usr/local/python3.5.2/lib/python3.5/site-packages/pydgraph/client_stub.py", line 51, in query
credentials=credentials)
File "/usr/local/python3.5.2/lib/python3.5/site-packages/grpc/_channel.py", line 824, in call
return _end_unary_response_blocking(state, call, False, None)
File "/usr/local/python3.5.2/lib/python3.5/site-packages/grpc/_channel.py", line 726, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1582799517.446788631","description":"Error received from peer ipv4:172.17.49.93:9180","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Socket closed","grpc_status":14}"

zero error msg(no alpha error msg):
W0227 18:31:57.050529 22426 raft.go:729] Raft.Ready took too long to process: Timer Total: 504ms. Breakdown: [{disk 504ms} {proposals 0s} {advance 0s}]. Num entries: 0. MustSync: false

I can provide data if needed.

[shekarm]

@EnglishVillage Yes, please provide the data. Meanwhile, we will investigate this as well.

[martinmr]

The error means that the client cannot connect to the alpha. If the alpha is up, you might be using the wrong destination.

I would look first at the code that creates the client and make sure it's the gRPC port of one of the alphas. Please note that Ratel uses the HTTP port, which is different. Maybe you are using the same port for both Ratel and pydgraph.

[EnglishVillage]

@martinmr
first,the rigth code is run.
in ratel I use port 8180,in pydgraph I use port 9180.
Because I increased the parameter of -o 100 in alpha start.

second,Report error every time the error code is executed.
so I think the wrong code is causing the alpha to crash.

[EnglishVillage]

@shekarm
g01.rdf.gz
g01.schema.gz

[EnglishVillage]

@shekarm
How are you testing,is this bug real?

[shekarm]

@EnglishVillage Yes, we are investigating it. Thanks.

[tizanae]

Running into the same details = "Socket closed" error when trying to set schema then push some data right after. I run an alter and the schema sets, but I get the socket closed immediately after with the mutate

[martinmr]

@tizanae That's probably a different bug. Feel free to open a bug report. The alpha logs would be helpful.

I downloaded the data and I could not complete the query because it times out. My guess is that you probably saw a OOM. This seems like an issue with the k-shortest paths algorithm or the graph is too big to fit into memory but that seems unlikely given the query is only asking for two paths. I'll mark this as accepted so someone can try to find where the issue is.

[all-seeing-code]

Hi @EnglishVillage numpaths=2 follows K-shortest Path algorithm which was just updated in this PR. Running your query against the above PR in Dgraph-Ratel returns the below noted output for numpaths=2. I think pydgraph error should also go away since I think it was timing out in the back-end before the fix.

  "data": {
    "all": []
  },
  "extensions": {
    "server_latency": {
      "parsing_ns": 132753,
      "processing_ns": 580219,
      "encoding_ns": 11095,
      "total_ns": 833189
    },
    "txn": {
      "start_ts": 9
    },
    "metrics": {
      "num_uids": {
        "": 1,
        "_total": 11,
        "base_company.name": 0,
        "base_company.parent_company_group_name": 0,
        "invest_capital.base_company": 5,
        "invest_capital.name_short": 0,
        "uid": 0,
        "xid": 0,
        "~invest_capital.base_company": 5
      }
    }
  }
}``` 

[all-seeing-code]

Closing this for now. Please re-open if the issue persists.