Solid reflection in rust

These "enterprise" patterns are usually bad practice, unnecessary and inefficient. The useful ones (e.g. serialization) are doable with proc macros for the parts of the program where they are needed.

Not necessarily. I won't call them a "bad practice". There is a strong need for configurable systems in enterprise use where each locality may have diverse business requirements. In fact DI is almost a standard issue these days for web-based enterprise systems.

These patterns may be less efficient than highly-optimized machine code, but not all applications require maximum CPU throughput. Many enterprise systems are data-driven and UI-driven and the bottleneck is usually not in computing resources.

18 Likes

In theory Rust can be compiled into JVM byte code (or corresponding C# thing), via something like this GitHub - davidar/lljvm: Low Level Java Virtual Machine . And restrict optimization to only basic one I suppose it would be possible to use reflection as is. May be it would require some kind of annotations, like this module is class and all functions in it are static methods to make work with reflection more simple.

Just wild imagination

If by DI you mean XML based configuration that mentions class names, that's obviously a bad practice since now renaming classes or removing apparently unused ones breaks the program, while a basic principle of well-designed programming languages is that any identifier can be renamed with no adverse effects and dead code can be removed.

Instead, you should just assign a string to each possible implementation and instantiate via a match statement.

If instead by DI you mean constructors that take instances of other objects, you can just do in Rust and there's no need to have a special name for the practice.

Not really. DI has nothing to do with XML-based configuration files with class names. And I'm not sure why you keep calling it a "bad practice" because, in practical enterprise software, people don't rename classes without very good reasons.

DI is a programming concept which automatically injects dependencies into a system based on the IOC (inversion of control) principle. It can be implemented as XML-based configuration files containing class names (in fact, that's exactly now common C# and Java DI containers work), however I consider that a minor implementation detail.

Many of my projects with DI have no configuration files at all. They are configured via code. So those "class names" are compiled in and are never wrong. That doesn't prevent it from being a DI project using a DI container framework.

6 Likes

When everyone is giving the same answers to you and you think their answers are all wrong, perhaps it is you that is not understanding. From your posts, I personally don't think you actually understand "Run-Time Reflection". You see it as a golden hammer that must exist in every language, when, in fact, it is something that exists in some dynamic languages to facilitate certain design patterns that are otherwise not representable at compile time.

The answers you are getting basically boil down to, "everything you think you want run-time reflection for, can be implemented more usefully at compile-time using macros/proc-macros and or existing language features". No one can give more specific answers because you haven't been specific about what you want to achieve. You're basically in the "X/Y problem (https://en.wikipedia.org/wiki/XY_problem)" feedback loop and you won't get anywhere by claiming that everyone else is to dumb to understand you.

22 Likes

Hi , This is the first answer very good. Thanks for the time dedicated for writing this long message. It is interesting. Surely i have to think a bit more along about all the specification about rust written but the first impression is: rust could give more effort for simplyfing this part even if you tell it is possible. Another point:

Blockquote hese patterns may be less efficient than highly-optimized machine code, but not all applications require maximum CPU throughput. Many enterprise systems are data-driven and UI-driven and the bottleneck is usually not in computing resources.

Really normally it is the opposite. The success of reflection is to permit high performance code : why? because iven if it is runtime, the time when you generate the runtime code is different from the time when is executed. The reflection permits to reduce the generic algorithm model reducing the input paramenters so you can write a more optimized code for the needs defined at runtime (but executed in after).You reduce the computational model. Normally reflection is executed when you load the server on startup .

1 Like

Well I think you're mixing up a lot of concepts. Reflection has nothing to do with what you're saying. Reflection is obtaining runtime information about types and behaviors. It has nothing to do with how the types are implemented and compiled.

In fact, I think you're really not talking about reflection at all. You're talking about monomorphization of generics.

There are large debates regarding whether generics monomorphization is better for speed or whether the functions themselves should stay generic and do runtime type casting. I am not sure the question is decided yet, but most of the fastest languages use monomorphization for generics. For example, C++ monomorphizes generics. Rust does of course. Even C# monomorphizes generics. The oddball out is probably Java...

On the other hand, just the fact that a generic function is monomorphized at runtime during JIT and others are monomorphized at compile time has nothing to do with reflection, and really has little to do with performance. Eventually they are all the same. Only the monomorphization is done at different times.

So I beg to differ from your comment. IMHO, reflection has NOTHING to do with high performance code. It is not about performance at all.

8 Likes

I think the word you are looking for is "monomorphization" not "monopolization".

1 Like

Thanks for catching the spelling error!

Or maybe about JIT compilation?

1 Like

Reflection, code generation, instrumentation are 3 aspects of the modern frameworks used also in collaboration. Jit is another tecnology present inside the compiler normally.

What do you mean by “modern framework”? The term sounds a bit unspecific.

In fact, I think you're really not talking about reflection at all. You're talking about monomorphization of generics.

One use-case that I've encountered a lot in Java was:

  1. Introspect the component (commonly, data model) to find its features and properties.
  2. Generate efficient accessors via API (like this, if I remember correctly) or even by generating bytecode directly.

It's also usually related to linking components at runtime, either due to requirements (plugins or loosely connected components) or simply because it's Java and everything is loaded dynamically, so why not?

I think, "Reflection" is an umbrella term for all of these together in Java land; it goes the full way from introspection to generating byte code (which is eventually compiled into native code by JIT).

An example would be to take some data model and compile XPath expression such that it avoids any indirection and behaves as it was written manually against types.

I sense that maybe this is what underlying question / use-case is here: how do we define our system as loosely connected components and assemble them into efficient runtime (although, this is much broader than "reflection", whatever that means). Which is a challenge in Rust we are seeing at my job here.

7 Likes

My good man, you really need to clear up your terminology. JIT definitely is NOT normally inside ANY compiler (at least where the term is used normally). It is typically for dynamic scripting languages to supplement an interpreter. In fact, I can say JIT is not in ANY compiler; it IS a compiler and is used in interpreters to compile bytecodes to machine code. Remember what JIT stands for -- Just In Time COMPILATION. A normal compiler is AOT (Ahead Of Time compilation).

I suggest you go and read up on the technical facts before raising an issue in a language internals forum.

9 Likes

And many static languages as well, such as Java and C#. They compile to a bytecode which is then interpreted/JITted by the runtime.

I agree that it's not "in" a compiler, and is instead part of the runtime, but I think you've been a little bit overagressive on the distinction.


The key point to Rust/LLVM's AOT though is that it goes ahead and produces the end result of JITting for all of the code, not just the hot code. Everything gets the same treatment as hot code under a JIT*.

* Modulo some dark magic around PGO where some JITs can actually optimize better than AOT without PGO because they can bias the JIT towards the observed workload.

4 Likes

Not an expert in Java, but looking at the references, it seems like a short-cut to create closures transforming one function/method signature to another while minimizing the overhead.

In C#, for example, we don't really need these. We just create new functions closing over the original reflected method, with acceptable overhead costs. Not sure why Java needs to have it specially, but it looks like they need it for performance reasons...

1 Like

What you mention is the debate between late-optimization with runtime profiles (e.g. in a multi-stage JIT) vs. PGO AOT compilation (as per @CAD97). I believe there is still a debate on whether runtime optimizations necessarily generate better code than PGO when the overhead of interpretation and dynamic compilation is factored in.

This, again, also has nothing to do with reflection. You don't need reflection for runtime-optimizations. It can even be done on machine code, with a background watcher collecting trace profiles and then dynamically re-optimize the instructions based on that. In other words, you don't need runtime type information for doing that.

So I'm quite sure now that you do not mean reflection.

What you're interested in is late-compilation from source with the ability to optimize based on runtime profiles in order to achieve higher speed than ATO compilation.

1 Like

Dynamically linked code could get better treatment under JIT as JIT will be able to optimize across modules. For example, it can inline trivial methods from code you loaded at runtime. Or you can generate code specifically for some script you need to run (say, JsonPath expression you obtained at runtime) against your data model, as if it was some code you wrote manually against your structs.

There are also some interesting things happening in GraalVM / Truffle in Java as well.

I think, this really gets into off-topic territory, though.

3 Likes

Honestly, this is a textbook case of a help vampire, and you should all probably stop encouraging them.

18 Likes