Skip to content

debuginfo: Fix gdbr-check expectations of formatting #40351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

amboar
Copy link

@amboar amboar commented Mar 8, 2017

This pull-req is one of the solutions outlined in #39838. There's a fairly detailed run-down there that I won't repeat here. I'm not sure which way the true solution is meant to go, so I'm creating a pull-req to give it a bit more attention.

This was a largely automated process, driven by the hack of a shell
script below.  The process to generate the patch was to repeatedly run
the script until the changes reached a fixed-point.

```bash

set -euo pipefail

O=$(mktemp)
./x.py test src/test/debuginfo 2>&1 | tee $O || true

awk '
function escape_pattern(pat,   safe) {
  safe = pat
  gsub(/[][^$.*?+{}\\()|]/, "\\\\&", safe)
  return safe
}

BEGIN { p = 0; n = 0; }

/---- \[debuginfo-gdb\]/ {
	if (p == 1) {
		printf("%s|%s|%s\n", src, error, expected);
	}
	src = $3;
	p = 1;
	n = 0;
	name = "";
}

n == 1 && $0 ~ name {
	expected = $0;
	n = 0;
}

/error: line not found in debugger output/ {
	error = substr($0,43);
	split(error, pieces);
	name = escape_pattern(pieces[1]);
	print "Found " name " in: " $0 > "/dev/stderr"
	n = 1;
}

END { printf("%s|%s|%s\n", src, error, expected); }
' $O |
while read l; do
	SRC="$(echo "$l" | cut -d'|' -f1 | sed 's/debuginfo-gdb/debuginfo/')"
	ERR="$(echo "$l" | cut -d'|' -f2)"
	V="$(echo "$ERR" | cut -d' ' -f1 | sed 's/\$/\\$/g')"
	EX="$(echo "$l" | cut -d'|' -f3)"
	if echo "$EX" | grep '^\$' > /dev/null; then
		echo $SRC
		sed -ri '/\/\/ gdbr-check:'"$V"' /s|(// gdbr-check:).*|\1'"$EX"'|' src/test/$SRC
	else
		# this is a bit questionable
		EX="$(echo "$l" | cut -d'|' -f2)"
		sed -ri '/\/\/ gdbr-check:'"$V"' /s|(// gdbr-check:).*|\1'"$EX"'|' src/test/$SRC
		echo $SRC: Failed to patch with: $l
	fi
done

echo $O
```

Fixes: rust-lang#39838
Signed-off-by: Andrew Jeffery <[email protected]>
@rust-highfive
Copy link
Contributor

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @aturon (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@TimNN
Copy link
Contributor

TimNN commented Mar 8, 2017

First of all thanks for the PR @amboar!

A few notes:

  • About a week ago the tests were passing for me with a gdb head build from November. Ideally I would like to know what caused the change in GDB output since then before this PR is merged.
  • The change in output seems like a regression to me: previously the gdbr lines were using rust like output, however now the output is C like (eg. () vs {} for tuples or : vs = in structs).

@TimNN
Copy link
Contributor

TimNN commented Mar 8, 2017

So I just checked with gdb 7.12.50.20170308-git and while I do get two debuginfo test failures they are entirely unrelated to this PR.

The output of gdb --configuration is as follows, maybe that is relevant:

   configure --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu
             --with-auto-load-dir=$debugdir:$datadir/auto-load
             --with-auto-load-safe-path=$debugdir:$datadir/auto-load
             --with-expat
             --with-gdb-datadir=/usr/local/share/gdb (relocatable)
             --with-jit-reader-dir=/usr/local/lib/gdb (relocatable)
             --without-libunwind-ia64
             --with-lzma
             --with-python=/usr
             --without-guile
             --with-separate-debug-dir=/usr/local/lib/debug (relocatable)
             --with-babeltrace

@amboar
Copy link
Author

amboar commented Mar 10, 2017

A few notes:

  • About a week ago the tests were passing for me with a gdb head build from November. Ideally I would like to know what caused the change in GDB output since then before this PR is merged.

Yeah, it's curious. My interest is more in understanding what is going on than necessarily getting this PR merged in its current form.

I still get the failures on recent Rust master (b04ebef) running in a Ubuntu Yakkety environment, with both the Ubuntu system gdb and a local gdb build from mid February.

@TimNN what environment are you testing in?

For the record the diff between the system and local gdb configurations is minimal:

$ diff -u <(/usr/bin/gdb --configuration) <(/usr/local/bin/gdb --configuration)                                                           [6/1883]
--- /dev/fd/63  2017-03-10 15:21:43.586582443 +1100
+++ /dev/fd/62  2017-03-10 15:21:43.586582443 +1100
@@ -1,16 +1,15 @@
 This GDB was configured as follows:
-   configure --host=x86_64-linux-gnu --target=x86_64-linux-gnu
+   configure --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu
              --with-auto-load-dir=$debugdir:$datadir/auto-load
              --with-auto-load-safe-path=$debugdir:$datadir/auto-load
              --with-expat
-             --with-gdb-datadir=/usr/share/gdb (relocatable)
-             --with-jit-reader-dir=/usr/lib/gdb (relocatable)
+             --with-gdb-datadir=/usr/local/share/gdb (relocatable)
+             --with-jit-reader-dir=/usr/local/lib/gdb (relocatable)
              --without-libunwind-ia64
              --with-lzma
-             --with-python=/usr (relocatable)
+             --with-python=/usr
              --without-guile
-             --with-separate-debug-dir=/usr/lib/debug (relocatable)
-             --with-system-gdbinit=/etc/gdb/gdbinit
+             --with-separate-debug-dir=/usr/local/lib/debug (relocatable)
              --with-babeltrace

Probably not surprising given they're both failing without the patch in the PR.

  • The change in output seems like a regression to me: previously the gdbr lines were using rust like output, however now the output is C like (eg. () vs {} for tuples or : vs = in structs).

So for me, without the patch in the PR the tests fail when the pretty printers from src/etc/*.py are loaded into gdb, and pass when we stop that from happening somehow (e.g. by emptying one of the python files as outlined in #39838, or if the gdb at hand wasn't built with python). That says to me that whatever the src/etc/*.py files are doing, it's not what the tests are expecting. If the test expectations are correct in their current form then src/etc/*.py need to be fixed; or, as this PR proposes, make the test expectations match the output on the assumption that src/etc/*.py are doing the right thing.

@TimNN it'd be interesting to know if your gdb was confirmed to load the files and pass the tests. In which case there's something else at play that should be tracked down.

@TimNN
Copy link
Contributor

TimNN commented Mar 10, 2017

@TimNN it'd be interesting to know if your gdb was confirmed to load the files and pass the tests. In which case there's something else at play that should be tracked down.

For most of the debuginfo tests, gdb is not supposed to load any pretty printers, they all use #![omit_gdb_pretty_printer_section] (The tests without this attribute were not changed in this PR so seem to pass for you).

I think the interesting question now is why your gdb does load the pretty printers.

@philipc
Copy link
Contributor

philipc commented Mar 12, 2017

I think I ran into this problem a while ago, but I can't remember the details. A couple of things to check:

  • Use objdump -j .debug_gdb_scripts -s <path> to make sure that the test binaries really have omitted the .debug_gdb_scripts section.
  • Have you configured rust with --enable-debug? If so, I think that might have been the cause.

@TimNN
Copy link
Contributor

TimNN commented Mar 12, 2017

I get objdump: section '.debug_gdb_scripts' mentioned in a -j option, but not found in any input file When attempting the objdump command from @philipc on eg. build/x86_64-unknown-linux-gnu/test/debuginfo/associated-types.stage2-x86_64-unknown-linux-gnu.

@TimNN
Copy link
Contributor

TimNN commented Mar 14, 2017

I have just reproduced this while investigating something unrelated, without further investigation I think this is related to either:

a) gdb being compiled for multiple targets or
b) rustc (or just tests?) compiled without optimizations

@amboar: Did you do one of those two things?

@amboar
Copy link
Author

amboar commented Mar 14, 2017

Sorry guys, I was trying to get back to this today but other things have intervened so far. Hoping to get time in the next few hours.

@amboar
Copy link
Author

amboar commented Mar 14, 2017

@TimNN I built two configurations:

$ ./configure
...
$ ./x.py build --incremental
...
$ ./x.py test --incremental src/test/debuginfo
...

This passed the debuginfo tests.

On the other-hand,

$ ./configure --enable-debug
...
$ ./x.py build --incremental
...
$ ./x.py test --incremental src/test/debuginfo
...

fails the tests.

I haven't looked into why yet but at least on ppc64 we need to run ./configure, otherwise ./x.py bails out:

$ uname -m
ppc64le
$ ./x.py
unknown cpu type: ppc64le
$ ./configure
...
$ ./x.py
downloading https://static.rust-lang.org/dist/2016-12-18/rust-std-1.14.0-powerpc64le-unknown-linux-gnu.tar.gz
                                                                           0.5%

I doubt --incremental is significant, but it was what I was doing. Further, I hadn't noticed the release mode passed the tests, as at the point that I was debugging the problem I was also looking into other issues that were already known to occur only in debug mode. Unfortunately I had overlooked #![omit_gdb_pretty_printer_section] as well, so apologies for not being very thorough!

@philipc
Copy link
Contributor

philipc commented Mar 14, 2017

I don't have a build to check on right now, but probably the debug libraries include the .debug_gdb_scripts section, and that ends up in the binary even if the test has #![omit_gdb_pretty_printer_section]. It would be worth running objdump on them to verify.

@amboar
Copy link
Author

amboar commented Mar 14, 2017

@philipc spot on!

From the release build:

andrewrj@andrewrj-guest:~/rust$ grep enable-debug config.mk
andrewrj@andrewrj-guest:~/rust$ echo $?
1
andrewrj@andrewrj-guest:~/rust$ objdump -j .debug_gdb_scripts -s build/x86_64-unknown-linux-gnu/test/debuginfo/associated-types.stage1-x86_64-unknown-linux-gnu

build/x86_64-unknown-linux-gnu/test/debuginfo/associated-types.stage1-x86_64-unknown-linux-gnu:     file format elf64-x86-64

objdump: section '.debug_gdb_scripts' mentioned in a -j option, but not found in any input file
andrewrj@andrewrj-guest:~/rust$ ldd build/x86_64-unknown-linux-gnu/test/debuginfo/associated-types.stage1-x86_64-unknown-linux-gnu
        linux-vdso.so.1 =>  (0x00007fff07de1000)
        libstd-dc2a73ed68c282a1.so => /home/andrewrj/rust/build/x86_64-unknown-linux-gnu/test/debuginfo/../../stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-dc2a73ed68c282a1.so (0x00007f0cdd26f000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0cdd04f000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0cdcc88000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0cdca84000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f0cdc87a000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0cdc65c000)
        /lib64/ld-linux-x86-64.so.2 (0x000056343c52e000)
andrewrj@andrewrj-guest:~/rust$ objdump -j .debug_gdb_scripts -s /home/andrewrj/rust/build/x86_64-unknown-linux-gnu/test/debuginfo/../../stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-dc2a73ed68c282a1.so

/home/andrewrj/rust/build/x86_64-unknown-linux-gnu/test/debuginfo/../../stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-dc2a73ed68c282a1.so:     file format elf64-x86-64

objdump: section '.debug_gdb_scripts' mentioned in a -j option, but not found in any input file

From the debug build:

andrewrj@andrewrj-guest:~/rust-debug$ grep enable-debug config.mk                                                                                                       [6/1804]
CFG_CONFIGURE_ARGS   := --enable-debug
andrewrj@andrewrj-guest:~/rust-debug$ objdump -j .debug_gdb_scripts -s build/x86_64-unknown-linux-gnu/test/debuginfo/associated-types.stage1-x86_64-unknown-linux-gnu

build/x86_64-unknown-linux-gnu/test/debuginfo/associated-types.stage1-x86_64-unknown-linux-gnu:     file format elf64-x86-64

objdump: section '.debug_gdb_scripts' mentioned in a -j option, but not found in any input file
andrewrj@andrewrj-guest:~/rust-debug$ ldd build/x86_64-unknown-linux-gnu/test/debuginfo/associated-types.stage1-x86_64-unknown-linux-gnu
        linux-vdso.so.1 =>  (0x00007ffe65d09000)
        libstd-1ad65c1c514eed75.so => /home/andrewrj/rust-debug/build/x86_64-unknown-linux-gnu/test/debuginfo/../../stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-1ad65
c1c514eed75.so (0x00007fd83d1cf000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd83cfaf000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd83cbe8000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd83c9e4000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fd83c7da000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd83c5bc000)
        /lib64/ld-linux-x86-64.so.2 (0x0000564293e40000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd83c2b3000)
andrewrj@andrewrj-guest:~/rust-debug$ objdump -j .debug_gdb_scripts -s /home/andrewrj/rust-debug/build/x86_64-unknown-linux-gnu/test/debuginfo/../../stage1/lib/rustlib/x86_64-u
nknown-linux-gnu/lib/libstd-1ad65c1c514eed75.so

/home/andrewrj/rust-debug/build/x86_64-unknown-linux-gnu/test/debuginfo/../../stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-1ad65c1c514eed75.so:     file format elf64-
x86-64

Contents of section .debug_gdb_scripts:
 4790a3 01676462 5f6c6f61 645f7275 73745f70  .gdb_load_rust_p
 4790b3 72657474 795f7072 696e7465 72732e70  retty_printers.p
 4790c3 79000167 64625f6c 6f61645f 72757374  y..gdb_load_rust
 4790d3 5f707265 7474795f 7072696e 74657273  _pretty_printers
 4790e3 2e707900 01676462 5f6c6f61 645f7275  .py..gdb_load_ru
 4790f3 73745f70 72657474 795f7072 696e7465  st_pretty_printe
 479103 72732e70 79000167 64625f6c 6f61645f  rs.py..gdb_load_
 479113 72757374 5f707265 7474795f 7072696e  rust_pretty_prin
 479123 74657273 2e707900 01676462 5f6c6f61  ters.py..gdb_loa
 479133 645f7275 73745f70 72657474 795f7072  d_rust_pretty_pr
 479143 696e7465 72732e70 79000167 64625f6c  inters.py..gdb_l
 479153 6f61645f 72757374 5f707265 7474795f  oad_rust_pretty_
 479163 7072696e 74657273 2e707900 01676462  printers.py..gdb
 479173 5f6c6f61 645f7275 73745f70 72657474  _load_rust_prett
 479183 795f7072 696e7465 72732e70 79000167  y_printers.py..g
 479193 64625f6c 6f61645f 72757374 5f707265  db_load_rust_pre
 4791a3 7474795f 7072696e 74657273 2e707900  tty_printers.py.
 4791b3 01676462 5f6c6f61 645f7275 73745f70  .gdb_load_rust_p
 4791c3 72657474 795f7072 696e7465 72732e70  retty_printers.p
 4791d3 79000167 64625f6c 6f61645f 72757374  y..gdb_load_rust
 4791e3 5f707265 7474795f 7072696e 74657273  _pretty_printers
 4791f3 2e707900 

@michaelwoerister
Copy link
Member

You could try filtering out the symbol name of the only symbol in the .debug_gdb_scripts for the linker script when #![omit_gdb_pretty_printer_section] is specified.

@amboar
Copy link
Author

amboar commented Mar 21, 2017

@michaelwoerister Thanks - I was considering something along those lines but hadn't looked at where to start. I'll follow your leads when I find some time to get back to the issue but it might not be for a week or so. If someone wants to pick it up in the mean time they should go for it.

@aturon
Copy link
Member

aturon commented Apr 13, 2017

I'm going to close out this PR for the time being, just to clear up the queue. @amboar, feel free to reopen (or open a new one) when you have a chance to try out @michaelwoerister's idea.

@aturon aturon closed this Apr 13, 2017
@amboar
Copy link
Author

amboar commented Apr 13, 2017

@aturon No worries

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants