Thank you for volunteering to be the guinea pig on this one @alexcrichton.
These logging crates are quite mature now. Seems like it’s about time to button them up and get them to 1.0.
First thing I’d like to do is write a list of example use cases for the cookbook, and actually write those examples. We can just edit them write into the op. Examples of existing statements for precedent here. I’ll help with these today. Feel free to just edit the ideas in the op directly.
Edit: I updated the op with some ideas for logging examples.
Yeah makes sense to me to cover env_logger here while we’re at it (as it’s in the same repo). It seems reasonable also while we’re taking a look at env_logger to maybe come up with guidelines (if possible) for implementing loggers.
@sfackler are you aware of any outstanding major issues against either log or env_logger? Or if anyone else has issues with these crates, please mention them!
What will be our relationship with alternative frameworks (e.g. slog) going forward? I’m concerned that we’ll have another rustc-serialize / serde split when a more advanced solution gains traction.
We can work around this via shims that forward from one framework to another, so it might not be as bad as it was with rustc-serialize.
On that note, what are our plans around structured logging? It seems like it’ll be a breaking change to add such a feature after-the-fact.
I ask because my employer enforces JSON formatted logs for all of its services. While we don’t have a case for Rust just yet, other users might have similar requirements.
For env_logger, can color support be added in a backward-compatible way? I guess this issue isn’t as important, since breaking changes here aren’t as disruptive as in the log crate.
That's a good question. Rust is going to face this situation continuously. There will always be lots of choices. I hope that the relationship between crate authors will be friendly.
When you say "our" I take that to mean "the rust project".
Personally, I want to encourage the most people to write the most and best Rust crates possible, for the ecosystem to be vibrant, and to evolve.
In the case of log for example, I doubt anybody is really in love with that crate, so I hope there will be other solutions. slog looks excellent and people should use it.
Regarding splits in the ecosystem, I think they are sure to happen, and we should expect it and expect to deal with it. We're still pretty much in the middle of the rustc-serialize -> Serde transition, but a rustc-serialize -> Serde post-mortem could make an enlightening thread. Do you see the rust-serialize / serde situation as problematic? My impression is that the situation has been pretty understandable and not so horrible, at least if one's expectations are suitably tempered.
I think we should do our best to walk a fine line between encouraging a proliferation of great Rust crates, and also working to make Rust crates more interoperable and discoverable. I think we can learn how to manage versioning and transitions in the ecosystem in a way that is reliable, and is also tolerant to change.
At least with respect to the log crate, I'd prefer not to add structured logging. My feeling about log is that it is mature and doesn't change much, so let's polish it up for 1.0 and be done. From there interested parties can regroup to think about future directions for logging in Rust. Maybe that leads to log 2.0, or a shift in focus to slog. That's just my opinion, as someone who hasn't maintained any of the log crates. @sfackler or @dpc might have opinions.
I suspect it can, but @sfackler would know better.
Anybody interested in helping out on the log crate effort? Don’t worry if it’s not clear to you what to do – @alexcrichton and others will help hook you up with work items. But please leave a comment if you’d like to be involved!
This is a context in which shims are a good way forward. There are some API design questions here about how to enable the shims to produce the various data structures (LogRecord for example) when performing the bridging, but it's pretty straightforward. IIRC slog already has crates to shim in both directions. This is a pretty standard approach in other languages as well (e.g. Log4j Bridge).
JSON-formatted output is basically orthogonal to the logging being structured or not. Structured logging is a pretty small extension on top of "normal" logging - just add on a parameter map to the log event state. See e.g. log4rs::encode::json - Rust for an example of JSON-formatted output from the standard log crate. All that would be added on top of that in a strucured logging world would be an extra params field.
Backwards compatibility is a bit wishy washy here since this isn't a Rust-level API. We could certainly add an extra control bit to turn on colored output, or even to it by default when hooked up to a TTY/console.
@brson[quote="brson, post:6, topic:5185"]
In the case of log for example, I doubt anybody is really in love with that crate
[/quote]
I think it's pretty solid. Is there any particular badness you're aware of?
Structured logging would not be a particularly large change on top of log - we could probably pull it off in a backwards compatible way right now if we wanted to. The only real design questions are what the structured log value type is (I just use Display for some stuff on an internal framework at work, slog uses a trait that allows you to get primitive types as well as strings) and what the log invocation syntax would look like.
Some other miscellaneous bits:
I'd like to remove the ability to shut down the logger. It exists because we used to try to do this with an atexit callback but turned that off because it deadlocks in some cases on Windows. With it gone, we can avoid two atomic add calls on each log event.
env_logger is a bit of a weird thing - It's basically just the rustc logger split out into its own crate when I rewrite log into a facade. I don't think anyone on the libs team has a particularly strong vision for what it would be. I'd like to spin it off out of the main log repo and let someone who does feel strongly take ownership over its evolution.
I’ve started to update the OP of this thread (remember, it’s a wiki!) with some of the discussion points coming up (thanks @lambda-fairy and @sfackler!)
@sfackler mind filling in the “Crate issues” section with any known blockers? (or just filling in “none”)
Otherwise if others would like to help out here with the log crate’s evaluation, there’s two primary entry points for doing so:
Volunteers for filling out the “Guidelines checklist”
Volunteers for creating some cookbook examples
I think for the guidelines checklist we can start out by claiming sections to start off. For example I’ll start out by taking the “Organization” section above and verifying those guidelines, namely C-REEXPORT and C-HIERARCHY. The task here is basically to go over the crate’s API and verify that the corresponding guideline is adhered to. Let’s fill in a [x] if the guideline is respected, and otherwise fill it in with [!] if the guideline needs some work to be fully respected. I’ll synthesize these later into notes for the actual review, and eventually they’ll become issues on the log tracker. For now it’s just checking some checkboxes!
If you’d like to help out with that, please feel free to just leave a comment here with what section you’d like to review, or simply fill in your name next to the section on the top. You’ll find my name’s listed next to the “Organization” topic above.
For cookbook examples recall that the intention here is to provide some substantive examples for working with crates through the ecosystem, focused on everyday tasks that end up going beyond the standard library. The cookbook can be found online for a few examples already. I’ve filled in the “Cookbook use case statements” of the OP with a few examples of examples we could write for log. If you’re interested in fleshing one of those out to a full-fledged cookbook example please feel free!
A good way to organize the cookbook examples I think are to use Integer32’s playground, for example here’s a small hello world. I figure we can basically turn the bullet points above into gists to the playground. If you’re interested in filling one of these out, please feel free to do so! (remember you can edit the OP as well!)
FWIW, I ended up making a pretty_env_logger because I wanted some color in the logs (and it tries to pad the logger name to the longest known name, but it's not perfect...). I could make some changes to allow turning those formatting options on, and/or detecting that stdout is a TTY, and release that as a env_logger 1.0 or something.
I want a custom log formatter that would return a new LogRecord instead of a String (I really have no idea how this could work, but I’m willing to help if it is possible)
Basically, I need to make as few allocations as possible for logging, I want a custom formatter and a custom target, and the current design did not help me there. I ended up copying a lot of code to do what I want, but I’d be happy to help log get there if possible
Now that https://github.com/rust-lang/rfcs/pull/1974 uses a trait, it’s tantalizingly close to mirroring exactly what would be needed to unify the std and no-std approaches with log. Not saying we need block 1.0 on this, but if we don’t then I hope log 2.0 is fine?
In designing allocationless formatting for fern*, I ended up with a callback solution. It’s not exactly the most ergonomic to use, but it isn’t too bad, and is efficient.
The one thing I would want is a way to create and modify LogRecords. Having a RecordBuilder which could build records from scratch, or modify only one property of an existing record with a From<LogRecord> impl would be super nice for both formatting and testing.
Having it as a builder would keep the ability to add new fields, and allowing use of it to modify an existing record would let one format the message of a log record without having to care about when nev fields are added.
This would also let log back ends have saner testing - right now I basically use cargo test -- --skip test2 and cargo test test2 to test two different configurations, since the only way to send log messages to a logger is having it as the global logger, and that can only happen once in a program.
Thanks for the link! I've added it to the discussion points above. My guess is there's not reason to not have support like this in env_logger itself, though!
Oh note that Argumentsimplements Copy, so you should be able to just use * to do the promotion.
Excellent constraint! I've updated the discussion topics above. In general I personally feel log is intended to be used in "no allocation" scenarios (e.g. the no_std availability), altough loggers built on top don't always expose all the functionality necessary for this (as you've found).
Awesome, thanks for the info! I'll update the topics above as well. Mind expanding a bit on the technical differences between fern and log as well? It seems related to the "how does slog relate to log" question as well!
Awesome thanks so much @gamazeps! If you've got any questions please feel free to check in here and ask. And if anyone else would like to also work on cookbook examples here, having multiple examples to work with is always a great "problem" to have!
Right, maybe I wasn't clear. I meant, since it sounded like some people wanted the feature, and it also sounded like maybe env_logger would be jettisoned from the log repo, that it wouldn't be hard for me to take the work I did there and merge it into some new such env_logger repo and publish.
