[CI] APM beats update job it is falling

[kuisathaverat]

The APM beats update job it is falling with the following error

[2020-02-06T09:08:11.469Z] + make clean check testsuite apm-server
[2020-02-06T09:08:13.380Z] 2020/02/06 09:08:13 Found Elastic Beats dir at /var/lib/jenkins/workspace/Beats_apm-beats-update_master/src/github.com/elastic/apm-server/_beats
[2020-02-06T09:08:13.380Z] /var/lib/jenkins/workspace/Beats_apm-beats-update_master/.magefile cleaned
[2020-02-06T09:08:19.971Z] 2020/02/06 09:08:18 Found Elastic Beats dir at /var/lib/jenkins/workspace/Beats_apm-beats-update_master/src/github.com/elastic/apm-server/_beats
[2020-02-06T09:08:19.971Z] >> check: Checking source code for common problems
[2020-02-06T09:08:25.391Z] # github.com/elastic/apm-server/vendor/github.com/gogo/googleapis/google/api
[2020-02-06T09:08:25.391Z] vendor/github.com/gogo/googleapis/google/api/annotations.pb.go:22:11: undefined: proto.GoGoProtoPackageIsVersion3
[2020-02-06T09:08:25.391Z] vendor/github.com/gogo/googleapis/google/api/http.pb.go:26:11: undefined: proto.GoGoProtoPackageIsVersion3
[2020-02-06T09:09:22.101Z] Error: failed running go vet, please fix the issues reported: running "go vet ./..." failed with exit code 2
[2020-02-06T09:09:22.101Z] make: *** [check] Error 1

[kuisathaverat]

well, the error now is other, it can not find the update-beats.sh script file.

18:01:59 runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
18:01:59 runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
18:01:59 + ./script/jenkins/update-beats.sh
18:01:59 /tmp/jenkins1498002165747409123.sh: line 2: ./script/jenkins/update-beats.sh: No such file or directory
18:02:00 runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<

[kuisathaverat]

with the update to modules the script ./script/jenkins/update-beats.sh is deleted, I guess it is no longer needed. This breaks a couple of jobs in the beats CI,
https://beats-ci.elastic.co/job/elastic+apm-server+7.x+beats-update/
https://beats-ci.elastic.co/job/elastic+apm-server+master+beats-update/
https://beats-ci.elastic.co/job/Beats/job/apm-beats-update/job/master/
this run that script for the 7.x and master branches, I've ask the team if we can rid of those jobs.

[kuisathaverat]

related to elastic/apm-server#3536

[kuisathaverat]

about the beats-update job, with the change to use make beats-update we lose the possibility to execute the job on PRs, basically go get github.com/elastic/beats/v7@3eaa9443d9fe575c09a850662d1db3b922ca00b4 can not grab the version of the beat if the commit is a PR commit, before of that we trick the git references to grab PR commits, I will see if it is possible to make the same with go get.

The following two issues point me that it is not possible directly
golang/go#33908
golang/go#31191

[graphaelli]

@kuisathaverat how does this work now? It looks like /run apm-beats-update will trigger the check - is that right? Do any changes automatically trigger this check?

[kuisathaverat]

how does this work now? It looks like /run apm-beats-update will trigger the check - is that right?

yes, that the comment trigger

Do any changes automatically trigger this check?

no, there is no such condition

[graphaelli]

Is there an issue open already to have this run periodically? This job was very helpful in the past to raise awareness of API changes that APM Server will have to incorporate.

[kuisathaverat]

It runs with every merge to master (and other branches) so it runs several times per day

https://beats-ci.elastic.co/job/Beats/job/apm-beats-update/job/master/

[kuisathaverat]

I think you are talking about this one #13371

[kuisathaverat]

run it periodically would not be necessary because it is integrated with the changes on master/releases branches, so any change that breaks the update is detected

[graphaelli]

Understood, this exchange was confusing for me:

Do any changes automatically trigger this check?

no, there is no such condition

I would have said this check is triggered with every merge to master instead, but I understand your response now.