Skip to content

add tcp-pinger for mailservers #1672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 15, 2019
Merged

add tcp-pinger for mailservers #1672

merged 2 commits into from
Nov 15, 2019

Conversation

jakubgs
Copy link
Member

@jakubgs jakubgs commented Nov 12, 2019

This is an initial version of implementation of an RPC call for checking latency of mailservers.

Partially resolves status-im/status-mobile#9394.

Status: READY

@jakubgs jakubgs self-assigned this Nov 12, 2019
@ghost
Copy link

ghost commented Nov 12, 2019

Pull Request Checklist

  • Have you updated the documentation, if impacted (e.g. docs.status.im)?

@status-im-auto
Copy link
Member

status-im-auto commented Nov 12, 2019

Jenkins Builds

Click to see older builds (107)
Commit #️⃣ Finished (UTC) Duration Platform Result
792e6f3 #1 2019-11-12 17:51:05 ~49 sec android 📄log
✔️ 792e6f3 #1 2019-11-12 17:51:09 ~50 sec linux 📦zip
792e6f3 #1 2019-11-12 17:51:39 ~1 min ios 📄log
d3c4ca0 #2 2019-11-13 11:07:32 ~32 sec android 📄log
✔️ d3c4ca0 #2 2019-11-13 11:07:53 ~50 sec linux 📦zip
d3c4ca0 #2 2019-11-13 11:08:27 ~1 min ios 📄log
aed9e15 #3 2019-11-13 12:02:03 ~27 sec ios 📄log
aed9e15 #3 2019-11-13 12:02:05 ~31 sec android 📄log
✔️ aed9e15 #3 2019-11-13 12:02:19 ~42 sec linux 📦zip
0409524 #4 2019-11-13 12:04:53 ~22 sec ios 📄log
0409524 #4 2019-11-13 12:05:00 ~31 sec android 📄log
✔️ 0409524 #4 2019-11-13 12:05:11 ~40 sec linux 📦zip
✔️ 5de9033 #5 2019-11-13 12:10:51 ~37 sec linux 📦zip
5de9033 #5 2019-11-13 12:11:09 ~57 sec ios 📄log
5de9033 #5 2019-11-13 12:12:16 ~2 min android 📄log
a68c19d #6 2019-11-13 13:15:15 ~31 sec ios 📄log
✔️ a68c19d #6 2019-11-13 13:15:19 ~35 sec linux 📦zip
a68c19d #6 2019-11-13 13:16:33 ~1 min android 📄log
c5a6bef #7 2019-11-13 15:56:58 ~33 sec ios 📄log
✔️ c5a6bef #7 2019-11-13 15:57:05 ~39 sec linux 📦zip
c5a6bef #7 2019-11-13 15:58:19 ~1 min android 📄log
9022e24 #8 2019-11-13 16:07:52 ~28 sec ios 📄log
✔️ 9022e24 #8 2019-11-13 16:08:05 ~40 sec linux 📦zip
9022e24 #8 2019-11-13 16:09:10 ~1 min android 📄log
15b5060 #9 2019-11-13 16:35:21 ~31 sec ios 📄log
✔️ 15b5060 #9 2019-11-13 16:35:31 ~40 sec linux 📦zip
15b5060 #9 2019-11-13 16:36:40 ~1 min android 📄log
f3c7f0d #10 2019-11-13 17:12:12 ~30 sec ios 📄log
✔️ f3c7f0d #10 2019-11-13 17:12:19 ~36 sec linux 📦zip
f3c7f0d #10 2019-11-13 17:13:31 ~1 min android 📄log
45f199b #11 2019-11-13 23:41:00 ~32 sec ios 📄log
✔️ 45f199b #11 2019-11-13 23:41:23 ~54 sec linux 📦zip
✔️ 45f199b #11 2019-11-13 23:48:57 ~8 min android 📦aar
c151e1d #12 2019-11-13 23:50:02 ~27 sec ios 📄log
✔️ c151e1d #12 2019-11-13 23:50:14 ~38 sec linux 📦zip
✔️ c151e1d #12 2019-11-13 23:56:12 ~6 min android 📦aar
✔️ 3486a31 #13 2019-11-13 23:58:59 ~37 sec linux 📦zip
3486a31 #13 2019-11-14 00:00:49 ~2 min ios 📄log
✔️ 3486a31 #13 2019-11-14 00:05:00 ~6 min android 📦aar
8c8675a #14 2019-11-14 00:49:28 ~18 sec ios 📄log
✔️ 8c8675a #14 2019-11-14 00:49:52 ~41 sec linux 📦zip
✔️ 8c8675a #14 2019-11-14 00:55:47 ~6 min android 📦aar
✔️ 8a4bf5d #15 2019-11-14 00:58:33 ~34 sec linux 📦zip
8a4bf5d #15 2019-11-14 01:00:41 ~2 min ios 📄log
✔️ 8a4bf5d #15 2019-11-14 01:04:54 ~6 min android 📦aar
2714fbb #16 2019-11-14 08:19:05 ~31 sec ios 📄log
✔️ 2714fbb #16 2019-11-14 08:19:16 ~41 sec linux 📦zip
✔️ 2714fbb #16 2019-11-14 08:25:18 ~6 min android 📦aar
b441eb8 #17 2019-11-14 08:25:18 ~25 sec ios 📄log
✔️ b441eb8 #17 2019-11-14 08:25:49 ~55 sec linux 📦zip
✔️ b441eb8 #17 2019-11-14 08:32:24 ~7 min android 📦aar
6501d5d #18 2019-11-14 09:48:39 ~42 sec ios 📄log
✔️ 6501d5d #18 2019-11-14 09:48:46 ~48 sec linux 📦zip
✔️ 6501d5d #18 2019-11-14 09:55:34 ~7 min android 📦aar
b230da5 #19 2019-11-14 10:21:54 ~23 sec ios 📄log
b230da5 #19 2019-11-14 10:22:01 ~32 sec android 📄log
✔️ b230da5 #19 2019-11-14 10:22:12 ~40 sec linux 📦zip
9c97c0c #20 2019-11-14 11:41:03 ~27 sec ios 📄log
✔️ 9c97c0c #20 2019-11-14 11:41:11 ~35 sec linux 📦zip
✔️ 9c97c0c #20 2019-11-14 11:47:03 ~6 min android 📦aar
✔️ 183094d #21 2019-11-14 11:46:02 ~30 sec linux 📦zip
183094d #21 2019-11-14 11:46:08 ~37 sec ios 📄log
✔️ 183094d #21 2019-11-14 11:52:46 ~5 min android 📦aar
9f11977 #22 2019-11-14 13:53:20 ~29 sec ios 📄log
✔️ 9f11977 #22 2019-11-14 13:53:33 ~41 sec linux 📦zip
✔️ 9f11977 #22 2019-11-14 13:59:45 ~6 min android 📦aar
✔️ be8e8f6 #23 2019-11-14 13:56:47 ~33 sec linux 📦zip
✔️ be8e8f6 #23 2019-11-14 13:59:52 ~3 min ios 📦zip
✔️ be8e8f6 #23 2019-11-14 14:06:55 ~7 min android 📦aar
✔️ 58bf486 #24 2019-11-14 14:39:35 ~1 min linux 📦zip
✔️ 58bf486 #24 2019-11-14 14:42:03 ~3 min ios 📦zip
✔️ 58bf486 #24 2019-11-14 14:45:48 ~7 min android 📦aar
✔️ 97db203 #25 2019-11-14 14:46:00 ~46 sec linux 📦zip
✔️ 97db203 #25 2019-11-14 14:48:07 ~2 min ios 📦zip
✔️ a4fe177 #26 2019-11-14 15:13:04 ~53 sec linux 📦zip
✔️ a4fe177 #26 2019-11-14 15:15:00 ~2 min ios 📦zip
✔️ a4fe177 #26 2019-11-14 15:17:54 ~5 min android 📦aar
✔️ 394846c #27 2019-11-14 15:17:38 ~48 sec linux 📦zip
✔️ 394846c #27 2019-11-14 15:19:21 ~2 min ios 📦zip
✔️ 394846c #27 2019-11-14 15:26:30 ~8 min android 📦aar
✔️ caa7ac3 #28 2019-11-14 15:48:01 ~40 sec linux 📦zip
✔️ caa7ac3 #28 2019-11-14 15:50:04 ~2 min ios 📦zip
✔️ caa7ac3 #28 2019-11-14 15:56:04 ~8 min android 📦aar
✔️ c299be7 #29 2019-11-14 16:08:20 ~41 sec linux 📦zip
✔️ c299be7 #29 2019-11-14 16:10:22 ~2 min ios 📦zip
✔️ c299be7 #29 2019-11-14 16:15:50 ~8 min android 📦aar
✔️ b475959 #30 2019-11-14 18:22:08 ~44 sec linux 📦zip
✔️ b475959 #30 2019-11-14 18:24:10 ~2 min ios 📦zip
✔️ b475959 #30 2019-11-14 18:29:01 ~7 min android 📦aar
✔️ 21b4b84 #31 2019-11-14 19:02:06 ~43 sec linux 📦zip
✔️ 21b4b84 #31 2019-11-14 19:03:56 ~2 min ios 📦zip
✔️ 21b4b84 #31 2019-11-14 19:08:21 ~7 min android 📦aar
✔️ b68dc98 #32 2019-11-14 19:44:29 ~37 sec linux 📦zip
✔️ b68dc98 #32 2019-11-14 19:46:33 ~2 min ios 📦zip
✔️ b68dc98 #32 2019-11-14 19:52:05 ~8 min android 📦aar
✔️ 153124f #33 2019-11-15 07:34:27 ~1 min linux 📦zip
✔️ 153124f #33 2019-11-15 07:36:08 ~2 min ios 📦zip
✔️ 153124f #33 2019-11-15 07:39:11 ~5 min android 📦aar
✔️ 69376e0 #34 2019-11-15 08:21:19 ~39 sec linux 📦zip
✔️ 69376e0 #34 2019-11-15 08:24:30 ~3 min ios 📦zip
✔️ 69376e0 #34 2019-11-15 08:27:16 ~6 min android 📦aar
✔️ 6bd667d #35 2019-11-15 09:08:37 ~44 sec linux 📦zip
✔️ 6bd667d #35 2019-11-15 09:10:52 ~2 min ios 📦zip
✔️ 6bd667d #35 2019-11-15 09:14:54 ~7 min android 📦aar
✔️ 476f01e #36 2019-11-15 09:30:21 ~45 sec linux 📦zip
✔️ 476f01e #36 2019-11-15 09:32:28 ~2 min ios 📦zip
✔️ 476f01e #36 2019-11-15 09:36:14 ~6 min android 📦aar
Commit #️⃣ Finished (UTC) Duration Platform Result
352fa0c #37 2019-11-15 11:48:11 ~29 sec ios 📄log
✔️ 352fa0c #37 2019-11-15 11:48:28 ~45 sec linux 📦zip
352fa0c #37 2019-11-15 11:49:46 ~2 min android 📄log
✔️ 88c4876 #38 2019-11-15 12:01:39 ~43 sec linux 📦zip
✔️ 88c4876 #38 2019-11-15 12:04:06 ~3 min ios 📦zip
✔️ 88c4876 #38 2019-11-15 12:06:33 ~5 min android 📦aar

@jakubgs
Copy link
Member Author

jakubgs commented Nov 12, 2019

Currently this doesn't seem to work as I'm unable to make and RPC request:

curl -s localhost:8545 -H 'content-type: application/json' \
    -d '{"jsonrpc":"2.0","method":"mailservers_Ping","params":[["1.1.1.1:53"], 1000],"id":1}' | jq
{
  "jsonrpc": "2.0",
  "id": 1,
  "error": {
    "code": -32601,
    "message": "the method mailservers_Ping does not exist/is not available"
  }
}

NOTE: Turns our the call name needs to be lowercase: mailservers_ping.

@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch 3 times, most recently from aed9e15 to 0409524 Compare November 13, 2019 12:04
@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch from 0409524 to 5de9033 Compare November 13, 2019 12:10
@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch 5 times, most recently from 15b5060 to f3c7f0d Compare November 13, 2019 17:11
@adambabik
Copy link
Contributor

Error in the CI:

[2019-11-13T17:13:30.666Z] # github.com/status-im/status-go/vendor/github.com/status-im/tcp-shaker

[2019-11-13T17:13:30.666Z] vendor/github.com/status-im/tcp-shaker/socket_linux.go:89:29: timeout.Milliseconds undefined (type time.Duration has no field or method Milliseconds)

@jakubgs
Copy link
Member Author

jakubgs commented Nov 13, 2019

Android seems broken, not sure why:

vendor/github.com/status-im/tcp-shaker/socket_linux.go:89:29: timeout.Milliseconds undefined

Something similar goes for iOS:

vendor/github.com/status-im/tcp-shaker/socket_darwin.go:92:29: cannot use timeout.Nanoseconds() (type int64) as type int32 in field value

@adambabik
Copy link
Contributor

vendor/github.com/status-im/tcp-shaker/socket_linux.go:89:29: timeout.Milliseconds undefined

This is added in Go 1.13 and I think we still use 1.12 in the CI.

vendor/github.com/status-im/tcp-shaker/socket_darwin.go:92:29: cannot use timeout.Nanoseconds() (type int64) as type int32 in field value

https://sourcegraph.com/github.com/golang/sys/-/blob/plan9/syscall_plan9.go#L258 it's clearly int32 so you need to do the conversion:

tsTimeout := unix.Timespec{Nsec: int32(timeout.Nanoseconds())}

@jakubgs
Copy link
Member Author

jakubgs commented Nov 13, 2019

Yeah, I get it, but why are you looking at a plan9 file?

@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch 5 times, most recently from 8c8675a to 8a4bf5d Compare November 14, 2019 00:57
@jakubgs
Copy link
Member Author

jakubgs commented Nov 14, 2019

Now I have no idea why iOS fails or the linting in tests...

@adambabik
Copy link
Contributor

[2019-11-14T01:00:09.599Z] vendor/github.com/status-im/tcp-shaker/socket_darwin.go:79:3: cannot use uint32(fd) (type uint32) as type uint64 in field value

[2019-11-14T01:00:09.599Z] vendor/github.com/status-im/tcp-shaker/socket_darwin.go:92:29: cannot use int32(timeout.Nanoseconds()) (type int32) as type int64 in field value

Problem with iOS. Sorry, I looked in a wrong file. But generally there are some issues with types so it just needs fixing as told.

And linter now... Did you update Go in the CI?

@jakubgs
Copy link
Member Author

jakubgs commented Nov 14, 2019

Same goes for Timespec, they have different types on arm and arm64:

type Timespec struct {
	Sec  int32
	Nsec int32
}

http://www.golang.so/src/syscall/ztypes_darwin_arm.go

type Timespec struct {
	Sec  int64
	Nsec int64
}

http://www.golang.so/src/syscall/ztypes_darwin_arm64.go

There is a helper method but it's not exported and it uses strict types:

func setTimespec(sec, nsec int64) Timespec {
	return Timespec{Sec: int32(sec), Nsec: int32(nsec)}
}

http://www.golang.so/src/syscall/syscall_darwin_arm.go

@jakubgs
Copy link
Member Author

jakubgs commented Nov 14, 2019

Found the solution to Timespect in golang/go#23078 which is the TimeToTimespec method:

func TimeToTimespec(t time.Time) (Timespec, error)

https://godoc.org/golang.org/x/sys/unix#NsecToTimespec

@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch 2 times, most recently from 9f11977 to be8e8f6 Compare November 14, 2019 13:56
@jakubgs
Copy link
Member Author

jakubgs commented Nov 14, 2019

@adambabik I managed to fix the build errors, but I have no clue what the linter issues are about.

Also, I tried building it with status-react via Nix but I'm getting:

rtt/rtt.go:31:24: elapsed.Milliseconds undefined (type time.Duration has no field or method Milliseconds)

Which makes no sense because the method clearly exists:

func (d Duration) Milliseconds() int64

https://golang.org/pkg/time/#Duration.Milliseconds
Gonna need to look into that too.

@jakubgs
Copy link
Member Author

jakubgs commented Nov 14, 2019

I think this issue - golang/go#28564 - indicates that the Milliseconds() method for time.Duration was added only in Go 1.13, and we are still on 1.12 in status-react:
https://github.com/status-im/status-react/blob/2000532c/nix/derivation.nix#L20
I have attempted to upgrade our nixpkgs to include 1.13 but it was quite difficult. For now I'm just going to do a simple calculation so we don't have to use that method.

@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch 9 times, most recently from 21b4b84 to b68dc98 Compare November 14, 2019 19:43
@jakubgs
Copy link
Member Author

jakubgs commented Nov 14, 2019

I ran a build of status-react on a branch called test-tcp-ping and it succeeded for all architectures:
https://ci.status.im/job/status-react/job/manual/516/
So we can consider this almost done, the only thing remaining is fixing the test/linting errors.

@adambabik
Copy link
Contributor

I have no idea why it started complaining about returned C types. In 153124f, this error is ignored. I reported this behaviour in golangci/golangci-lint#858. There is a simple case how to reproduce it.

@jakubgs
Copy link
Member Author

jakubgs commented Nov 15, 2019

@adambabik thanks for fixing that, then I guess we are ready to merge unless you think there's anything else this might be missing.

I added a small rtt/README.md to explain usage. Seemed like the right place to put it.

@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch from 153124f to 69376e0 Compare November 15, 2019 08:20
@jakubgs jakubgs changed the title [WIP] add tcp-pinger for mailservers add tcp-pinger for mailservers Nov 15, 2019
@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch from 69376e0 to 6bd667d Compare November 15, 2019 09:07
@jakubgs jakubgs removed the request for review from cammellos November 15, 2019 09:12
@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch from 6bd667d to 476f01e Compare November 15, 2019 09:29
@jakubgs jakubgs force-pushed the mailserver-tcp-pinger branch from 352fa0c to 88c4876 Compare November 15, 2019 12:00
@jakubgs jakubgs merged commit 749dae1 into develop Nov 15, 2019
@delete-merged-branch delete-merged-branch bot deleted the mailserver-tcp-pinger branch November 15, 2019 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Select mailservers based on ping
3 participants