Dtrace providers for rustc

[nikomatsakis]

So I’ve been thinking a lot about profiling and understanding the compiler recently. For example, I added a simple hack to enable profiling rustc queries with perf, which I’ve found very useful. But that was just based on a regexp applied to the symbol names, it’d be cool to be able to do deeper introspection.

Recently I stumbled across the concept of dtrace “application providers” – I was thinking it’d be really neat to integrate some of these into rustc! For starters, we could trigger on rustc queries, but we could also trigger for other kinds of events (e.g., red/green for incremental, etc). I’ve not looked that deeply yet, does anybody have experience with these?

(There are also language providers that rustc could emit to help users get more insight.)

[cuviper]

A while back I wrote a little macro for placing SystemTap probe points on Linux. In theory it could be more general, implementing DTrace probes for other unix-y platforms, or Microsoft’s ETW, but I didn’t get around to that. I had started this as rust PR14031, and while it wasn’t considered something that the standard library would want to commit to, it could still be worth internal rustc use.

cuviper/rust-libprobe

@Fiedzia also ported the idea into a compiler plugin:

Fiedzia/rust-usdt

[cuviper]

Oh, also, nowadays perf probe also understands SystemTap probe points, as does the newer bcc tool, so this has become a pretty standard way to add static annotations on Linux.

edit: GDB 7.5+ also supports these – I have an example in the docs here.

[nikomatsakis]

Forgive my ignorance @cuviper, I’ve not had time to do my background reading, but I have a question:

rustc queries have the form name(key) where the key is some data structure in the compiler. I would very much like to be able to do some sort of dtrace probe in which we expose this data structure to the probe, so that I can ask questions like “how many unique keys were there” or “what %age of time is spent doing any sort of query associated with this key”.

(We could certainly serialize all keys to strings.)

I guess what I’m asking is: what kind of data can I pack into one of these things?

[nikomatsakis]

Alternatively, what is the best summary for me to read?

[nikomatsakis]

OK, wait, I’m reading your crate docs now. I realize that my questions might have been answered had I clicked a bit more.

[nikomatsakis]

I guess what I’m asking is: what kind of data can I pack into one of these things?

From the rustdoc:

Any expression which can be cast as i64 is allowed as an argument. The arguments might not be evaluated at all when a debugger is not attached to the probe, depending on the platform implementation, so don’t rely on side effects.

Hmm. Not as general as I had hoped, but maybe ok.

[cuviper]

The underlying format is pretty limited, owing to its asm nature. You can pass {1,2,4,8}-byte values, signed or unsigned. The fact that rust-libprobe always uses i64 is because we don’t know types in macro_rules!, but I think the rust-usdt plugin does use more specific sizes.

For more complicated types, the easiest thing to do is pass a pointer, and then dereference that as needed on the tool side.

[nikomatsakis]

The underlying format is pretty limited, owing to its asm nature. You can pass {1,2,4,8}-byte values, signed or unsigned. The fact that rust-libprobe always uses i64 is because we don’t know types in macro_rules!, but I think the rust-usdt plugin does use more specific sizes.

I mean I sorta expected that, but I was hoping for some magic.

[nikomatsakis]

That said: the single most common sort of query key in rustc is a DefId, which is a pair of 32-bit integers, so integral values are fine for that (but we would need to provide some utilities to map from user values).

Another idea I had is to permit user-specified filters as command line arguments. So, we would install probes for each kind of internal query, and by default their argument would be 0. But you count pass filters that test against the keys – each filter gets a number, and if the filter matches, then we pass that value as the argument.

[cuviper]

Some of that type “magic” is recoverable on the tool side. With debuginfo, systemtap can do something like @cast($arg1, "YourType")->field.... It’s also possible to write “tapset” libraries to abstract this sort of thing.

[cuviper]

RE filtering - if you can present the appropriate arguments to the probe, then users can do custom filtering on the tool side. The GDB example that I linked uses condition 1 $_probe_arg1 > 1000.

[Fiedzia]

I guess what I’m asking is: what kind of data can I pack into one of these things?

This 64bit data can be a pointer, so you can pack anything you want. Any interpretation of the data is up to the client, you can use dtrace/systemtap/bpf code to interpret it in any way you need, so this can be a pointer to a struct or whtever and you can get anything that’s inside.

[Fiedzia]

I mean I sorta expected that, but I was hoping for some magic.

The magic is performed by the code you submit to kernel when you read probe data.

[nikomatsakis]

The magic is performed by the code you submit to kernel when you read probe data.

Does this code execute synchronously with rustc? How does it know that the pointer is still valid? Do you have an example of a client that accesses non-trivial data?

[Fiedzia]

The whole thing is managed by operating systen and the code is executed synchronously in the context of the process, so you have full access to its memory. Some example for bcc is here: https://github.com/iovisor/bcc/blob/master/docs/reference_guide.md#6-usdt-probes however all the existing tools expect c conventions, so they would need to be adapted for Rust.

[nikomatsakis]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.