Compiler Profiling Survey

[wesleywiser]

@Eh2406

Overall the question I would like a tool to answer is "what lines in my source file is the compiler spending its time on?" or as it is spending most of the time in llvm "what lines in my source file is generating the most llvm-ir?"

@alexcrichton

I'd echo @Eh2406's sentiment where being able to "blame" lines of code would be amazing . I'd love to see, for example, that 10% of our compile time is all from format! or something like that. From the perspective of a crate author they typically don't have much control over what the compiler is doing, so knowing what they can do to make the compiler's job easier would be a godsend. I realize though that this is super difficult, and likely still a long way out (especially because LLVM is where a lot of the interest lies and I'm not sure LLVM has great support for this)

Thanks for that feedback! This is definitely something we've heard before from other people. As you've said, it's not an easy problem I believe @michaelwoerister has mentioned that getting this kind of information out of LLVM is possible, but I don't know the details. The larger question we have is "what to actually do with this data?". The incremental compilation support in rustc gives us very fine-grained information about where time is being spent but we don't know how to aggregate that data. One particularly tricky question is how to deal with the effects of the query cache. For example:

fn foo(x: Vec<u8>) {
  ...
}

fn bar(y: Vec<u8>) {
  ...
}

During compilation, we'll need to calculate information about the type of x in foo() so we'll issue a query like layout_of(Vec<u8>) -> TyLayout which will take some amount of time to complete. The results of this query are then cached in memory. Later when we process bar(), we'll issue another query to get information about y, layout_of(Vec<u8>) -> TyLayout but this time, we already know the result and return it from the cache making it basically "free".

So, if we take the simplest approach to blaming time, we'll report foo() as being costly to the user while bar() is "zero" time. What's even more confusing is that if the user removes foo(), now suddenly bar() takes additional time to compile.

We're very interested in building good tooling which can answer the "what parts of my code are expensive to compile?" question, but we're not really sure how to get there. For this particular thread, we'd like to focus on the "profiling rustc" use case. If you have thoughts or ideas(!!) about profiling user code's compilation time, we'd really like the feedback on this GitHub issue.