x/build/cmd/rundockerbuildlets: scaleway buildlets fail looping #22748
Comments
|
Change https://golang.org/cl/78032 mentions this issue: |
Fixes golang/go#22592 Updates golang/go#22749 Updates golang/go#22748 Change-Id: I6b58f21ececa212a12af8ef21cdef9c90515b837 Reviewed-on: https://go-review.googlesource.com/78032 Reviewed-by: Andrew Bonventre <[email protected]>
|
I nuked all 50 instances and recreated and now I see them just deleting & restarting the containers: Why? Maybe because of #22749 is making the containers start up too slowly? |
|
Hm... archive/tar changes? The docker logs are saying: Note the I probably built the linux-arm binary with Go tip when I addressed #21839, and now the new tip-built buildlet can't parse the coordinator's Go 1.8 tar files. /cc @dsnet |
|
What's the code that generates the tar file (or reads it)? The "0 files, 0 dirs" is also interesting. |
|
The The tar.gz comes from x/build/cmd/coordinator which gets it either from gitmirror (which just does There's only one use of tar.NewWriter that we're using in the coordinator (to generate a VERSION file), but that push works fine. You can see it in the log above (the first one, |
|
Btw, I re-pushed a new buildlet binary built at Go 1.9 and they're all working again. So it does seem to be something at tip. |
|
Is there any way to get access to the tar.gz file being passed in? |
|
With some work I suppose. |
|
I have a theory. I added support for parsing the PAX records for an esoteric feature of tar, which are global PAX records, denoted by a typeflag of Assuming this is the issue, you probably want to either push the |
|
Confirmed. I ran You can see a typeflag of 'g' at offset 0x9c. |
|
Do you think the Go tar reader should expose these records by default, or do you think they should be opt-in? |
|
They were actually always returned in Go1.9 and earlier, but returned a bogus file containing the raw contents of the PAX headers. The buildlet logic actually happened to work by chance. There aren't any hard restrictions on what filename to use for global headers, so the one used by git just happened to work for I was actually surprised that the buildet worked at all in Go1.9 and earlier. The surprising behavior is that Unfortunately, I don't know how to fix |
|
Change https://golang.org/cl/78355 mentions this issue: |
|
Change https://golang.org/cl/78546 mentions this issue: |
Several usages of tar (reasonably) just use the Header.FileInfo
to determine the type of the header. However, the os.FileMode type
is not expressive enough to represent "files" that are not files
at all, but some form of metadata.
Thus, Header{Typeflag: TypeXGlobalHeader}.FileInfo().Mode().IsRegular()
reports true, even though the expected result may have been false.
To reduce (not eliminate) the possibility of failure for such usages,
use the placeholder filename from the global PAX headers.
Thus, in the event the user did not handle special "meta" headers
specifically, they will just be written to disk as a regular file.
As an example use case, the "git archive --format=tgz" command produces
an archive where the first "file" is a global PAX header with the
name "global_pax_header". For users that do not explicitly check
the Header.Typeflag field to ignore such headers, they may end up
extracting a file named "global_pax_header". While it is a bogus file,
it at least does not stop the extraction process.
Updates #22748
Change-Id: I28448b528dcfacb4e92311824c33c71b482f49c9
Reviewed-on: https://go-review.googlesource.com/78355
Reviewed-by: Brad Fitzpatrick <[email protected]>
Run-TryBot: Brad Fitzpatrick <[email protected]>
TryBot-Result: Gobot Gobot <[email protected]>
The scaleway builders are all saying:
/cc @jessfraz @adams-sarah @andybons
The text was updated successfully, but these errors were encountered: