Rust Debugging Quest


#1

I’ve recently started working full time to improve Rust debugging. I thought I’d post a bit about what I’m planning to do. I’m open to additions and commentary; and also I have some questions.

The main thing I’m doing first is writing a Rust plugin for lldb. This is needed in order to enable future improvements to debuginfo (see below for a case in point). I’m currently doing this work here, but at some point I suppose this should be moved to rust-lang-nursery. (Sooner or later? The first question for you.)

The plugin is maybe halfway done, with the remaining piece being expression parsing. For this I’m currently planning write an external parser using the syn crate (or perhaps libsyntax 2.0), and have lldb call into it.

For a while, I think the Rust community will have to ship a fork of lldb. This message on lldb-dev explains why. Even once the Rust plugin lands upstream, we’ll still need to ship the external parser. I talked a bit with acrichto about this, see the bug he filed.

An lldb plugin will let us evolve the debuginfo output of the compiler in parallel with fixes to the debuggers. I can write up a full list if it’s needed (many of the to-do items are in the github issues already), but some highlights are:

  • Correctly represent optimized enums. See this issue. I’ve actually implemented all of this for llvm (patches landed), rustc, and gdb – but I am having second thoughts about landing it. On the one hand, the enum optimization improvements regressed debugger support for this in some cases. On the other hand, these patches will make enums basically incomprehensible to lldb. So: land them anyway? Or hold them until the lldb plugin is done?

  • Emit debuginfo for traits. This will enable method calls via traits, and also operator overloading support.

  • Emit more debuginfo for virtual tables, allowing calls on trait objects.

  • Macro stepping. (This one doesn’t actually need the lldb port, it’s just something that keeps coming up…)

I’ll also be filing bugs against DWARF to try to get Rust bindings into the standard. There are already a couple of Rust-inspired extensions in LLVM’s DWARF output which should probably be standardized; and we’ll need some new tags as well.

One unknown is how to deal with Windows. There’s some PDB support in LLVM but it’s unclear to me how much of Rust can be expressed there.


#2

I say land them anyway. We’re going to have some churn either way, it’s okay if stuff is unstable for this year as long as we eventually settle down.


#3

Great work, @tromey! I think this will be much appreciated once the community can get their hands on it.

Some questions:

  • You say you have implemented the changes around optimized enums in LLVM, rustc, and gdb. Am I correct in assuming that only the LLVM part has landed upstream while the rustc and gdb changes are on still on personal branches that you maintain?
  • Is there a way to make gdb support both, the current format and the new one?

Depending on the answers to the question above I have two suggestions on how to move forward:

  1. If gdb cannot support both formats, we could:
    • keep the GDB changes in a fork while we still need to support the old format for LLDB,
    • make the compiler support both formats, with the new format only being emitted when a -Zdebuginfo-2018 flag is specified.
    • add a job to our CI that uses the GDB fork and tests the new format,
    • while regular users we keep using the current format.
  2. If we can make GDB support both formats,
    • we’d just land the changes in GDB,
    • land the changes in the compiler, but still behind a nightly-only flag,
    • update our tests to test both the new and the old format.

I’d like to not break current LLDB, especially as long as we don’t have an ETA for the proper Rust plugin. People are using it as far as I can tell. However, I would be fine with starting out with a minimal LLDB plugin that just maintains the level of support that we have now (i.e. no parser, no expression evaluation). If we ship that then we could just switch the compiler over to the format.

One question though is: What do we do about older versions of GDB out there? Would we just tell people “download a recent version of GDB”?


#4

Yes. The rust compiler patch is here and the gdb patch is here. I’m going to submit the gdb patch upstream soon, like probably today.

Yes, that’s what i implemented – the patch moves the current enum-handling bits into the DWARF reader, and then adds support for the new format there. So now the rust code in gdb only sees a single representation.

This is interesting. I was under the impression that you could currently evaluate expressions in LLDB, but you had to use C++ syntax. Is that not the case? Because my current Rust plugin for LLDB is basically DWARF reading without expressions – not 100% done, but close to that milestone.

Yes, I think that’s the only way. But also see this issue about shipping lldb. One idea would be to also sometimes ship our own gdb. I’ve argued against this in the past, but maybe it really is the way to go.


#5

OK, that sounds great!

That’s probably the case, yes. However, I don’t think one can call functions in a reliable way. The most important thing, I think, is that one can inspect the contents of variables. I guess that goes through expression evaluation as soon as you follow a pointer…

I think that’s acceptable.

So, do you agree that we should make the compiler support both the old and the new format, keep emitting the old format for now, but start testing the new format on our infrastructure? People on nightly Rust and with a recent GDB version could already start benefiting from it.


#6

Excellent news - this really really needed doing but I wasn’t the right guy to do it. Happy to help test out custom lldb builds asap. Anything I can do to help please let me know.


#7

It’s too soon to try it yet but I will definitely post here when it is ready. And, thank you.


#8

I recently discovered that the debug info for enums is already pretty broken in 1.24, see this LLVM bug. Maybe this can be fixed in some minimal way; but overall I’d rather just try to land my real enum patch; I’m wondering what you think about this.


#9

I think the Rust plugin is complete enough for adventurous users to try it out. It can parse and evaluate many Rust expressions. Function calls work. There are a few known bugs, a few things that could be polished, and of course probably a bundle of undiscovered bugs.

Scared off yet? If not, please give it a try. You can find build instructions here.

PRs and bug reports are gratefully accepted.


#10

Superstar! This is most excellent news.


#11

@tromey I wonder if it s possible to upstream the enum-debuginfo branch very soon? It somehow boggles me how people are actually debugging their rust binaries without proper debugging information, how are they able to inspect the enum values? Right now it’s nearly impossible (rust-nightly, gdb, lldb).


#12

Yeah, I am fixing up this branch now. Today I rebased it and fixed the problems. You can follow the progress in the bug: https://github.com/rust-lang/rust/issues/32920#issuecomment-412296203


#13

@tromey Thanks for the update! It seems that https://github.com/rust-lang/rust/pull/52716 has been merged.

But I am unable to install it with rustup component add lldb-preview --toolchain nightly although it seems that the package available in the toolchain: https://static.rust-lang.org/dist/2018-08-17/index.html (lldb-nightly-x86_64-apple-darwin.tar.gz)

Is this intended behaviour?


#14

No, it’s not. I don’t know what is missing yet (I didn’t look today) but you can follow along in this bug: https://github.com/rust-lang/rust/issues/48168