Crate evaluation for 2017-05-16: log

[gyscos]

Also interesting would be a way to log differently to the file and stdout (colored log to stdout, simple/structured/json to file)

[sfackler]

Sure - that’s a pretty standard setup with something like log4rs.

[daboross]

@alexcrichton

Awesome, thanks for the info! I'll update the topics above as well. Mind expanding a bit on the technical differences between fern and log as well? It seems related to the "how does slog relate to log" question as well!

Sure! I don't mean to bring it in to the discussion too much, but fern is a log-crate backend that I've been working on, similar to log4rs - but focusing primarily on simple builder-based configuration.

I mention it just because @Geal was talking about format_args!() and a custom log formatter without allocating intermediate Strings, and I've recently added this feature to fern. In fern, the user provides a closure which always ends with a callback like out.finish(format_args!(...)) rather than returning anything.

This way no Strings are allocated intermediately, and format_args!() can still use newly-created data for the format, like the current time. Using this callback style, no changes are needed in the log crate in order to support allocation-less log formatting.

@Geal

I didn't write this before, but I would be really interested in how you were wanting to implement format_args!() formatting! When I tried to do anything storing or returning an fmt::Arguments, I couldn't get it to succeed - I'd be interested to know if it can work better with a args() method returning fmt::Arguments.

Even storing a simple fmt::Arguments in a variable failed for me:

extern crate chrono; 
fn main() {
    let timefmt = chrono::Local::now().format("%H:%M:%S");
    let formatted = format_args!("{}", timefmt);
}
// error: borrowed value does not live long enough

(playpen)

The only workaround I found was using the result of format_args!() in a callback.

[dpc]

slog provides bi-directional log compatibility in slog-stdlog. It’s quite simple and ergonomic (IMO) so I don’t see a problem with slog and log coexisting for time being. There is no incompatibility like with serde/rustc-serialize since simple adapters can convert structured log record into plain record message, and plain text message is just a special case of a structured log record.

Right now it’s just worth considering if there will be no problems upgrading the ecosystem in case we ever start working on hypothetical log2 or something like that. We don’t want to split the ecosystem into “crates using old logging” and “crates using new logging”. Seems to me that that log2 compiled with the right features would just register itself as a backend for log and potentially provided a slog::Drain implementation and could interoperate with both. So no problems there that I can see.

[chrivers]

My very first published crate is actually for making colored log output, and multi-line support. I’m going to publish a version with more flexibility soon. Feedback extremely welcome!

https://crates.io/crates/colog

Oh, and the multi-line formatting is something I’d like to see in the standard logger as well.

[dbrgn]

@sfackler

Structured logging would not be a particularly large change on top of log - we could probably pull it off in a backwards compatible way right now if we wanted to.

I would really like to see structured logging in the log crate.

There are many libraries around that use the log crate, but where the log message are either too verbose or too generic. By providing an interface for structured logging (just a way to add arbitrary key-value pairs) we allow logging crates to use this information to filter and/or display the information in a meaningful way. Simple string messages are hard to parse and hard to filter.

Features like colors, multi-line logging etc are all great and nice-to-have, but they only affect the log representation. Before we focus on that, we should make sure that the log capture is actually solid and mostly feature-complete.

I also have the feeling that it shouldn't be too hard to add structured logging on top of the current API, although I didn't look at the codebase too closely.

@Geal

• I'm eagerly awaiting a release with that commit

Looks like I'm not the only one refreshing the commit list daily

Edit:

@alexcrichton

Should env_logger stay in the repository or should it get split out?

I think env_logger could benefit from living in a separate crate, with an independent release cycle.

[budziq]

@alexcrichton

Awesome thanks so much @gamazeps! If you've got any questions please feel free to check in here and ask. And if anyone else would like to also work on cookbook examples here, having multiple examples to work with is always a great "problem" to have!

Hi, I do not know if this is the place for such questions/suggestions.

I would be really eager to participate in the cookbook but as someone completely new to rust, its community and discourse I find it currently a little hard to approach (even though I have already submitted few examples) .

What is the canonical way to submit cookbook ideas and notify about ones willingness to implement them? Is it via replying to lib-blitz discourse threads or via github issues as mentioned in CONTRIBUTING.md?

I for one would opt for a centralized github based approach similar to the already created Url crate tracking issue: https://github.com/brson/rust-cookbook/issues/40. It makes it a little more approachable for newcomers like myself and helps to avoid work duplication (I would not know about log examples being ready for implementation if I did not stumble here by chance).

I am really looking forward to helping out and learning rust along the way!

[Nemo157]

While structured logging is nice, the big thing I can see that slog provides over log is contextual loggers. Even more importantly these drastically affect the API surface. From slog's docs:

• contextual - slog's Logger objects carry set of key-value data pairs that contains the context of logging - information that otherwise would have to be repeated in every logging statement.

I have to admit, I have not used either of these crates, trying them both out is still on my todo list. But I have worked with contextual loggers in the past in a C# web application, and found them super useful in terms of being able to inject data into log lines from other libraries. At the start of a request a logger containing the current request id is created, then that is passed everywhere that might log during the request so all log lines can be correlated with the request. Especially when using a multi-threaded service with async I/O this is really helpful as an individual request would bounce around between threads as different async operations complete.

[brson]

I’ve created a cookbook issue for log examples, and posted a PR with placeholders for them.

I did not create a placeholder for the syslog example because I don’t know which crate implements log for the syslog.

I also updated the op to include an issue for adding links to crates that implement the log interface to the README and crate docs. The only logging implementation I actually know of is env_logger!

[luser]

@Nemo157

I have to admit, I have not used either of these crates, trying them both out is still on my todo list. But I have worked with contextual loggers in the past in a C# web application, and found them super useful in terms of being able to inject data into log lines from other libraries. At the start of a request a logger containing the current request id is created, then that is passed everywhere that might log during the request so all log lines can be correlated with the request. Especially when using a multi-threaded service with async I/O this is really helpful as an individual request would bounce around between threads as different async operations complete.

We're using log + env_logger in sccache, and having some sort of contextual logger would be incredibly useful. The sccache server uses tokio and can handle multiple requests at a time, so we wind up prefixing all log messages with the object file or crate name being compiled, like so. Since we're using Futures all over the place we wind up cloning that String everywhere. It would be nice to have a more straightforward way to do that.

[sfackler]

We should certainly make sure the structured stuff is designed such that this can be done in a separate crate. I want to keep log itself as minimal as possible - it should be very low cost for any library crate anywhere to depend on it and log things.

[alexcrichton]

Ok I’ve filled out the evaluation template for C-REEXPORT and C-HIERARCHY, turned out to be not too hard as there’s no public modules in the log crate!

[brson]

It’s not clear to me whether some incomplete evaluation tasks have been claimed or not, so I added a section to the end of the template for listing tasks that people have volunteered for, and I added it to this op, and signed myself up for the “naming” and “interop” sections, which I will do shortly.

I also think that for simplicity’s sake we should stick to the log crate here, and worry about env_logger later. Otherwise we basically have to do the evaluation twice and create a whole second checklist. I’ll update the blitz thread to reflect this.

[brson]

I did the “naming” and “interop” sections and updated the op with the following:

Crate issues

• Implement Debug for LogMetadata, LogRecord
• FromStr impls for LogLevel, LogLevelFilter use () as error type

Guidelines changes

• C-CONV-TRAITS should discuss FromStr

Discussion topics

• Can any of these Debug representations be empty, per C-DEBUG-NONEMPTY?
• Possible trait impls
  • LogLocation: Eq/PartialEq, Ord/PartialOrd, Hash, Clone
  • LogMetadata: Eq/ParialEq, Ord/PartialOrd, Hash
  • LogRecord: Eq/PartialEq, Ord/PartialOrd, Hash
  • LogLevel: Hash
  • LogLevelFilter: Hash
• Should anything here implement serde?
  • If not what does that say about the guidelines C-SERDE?

[brson]

I’m going to do the docs checklist next.

[brson]

I also did “debuggability”.

[brson]

Updated the op with the following for documentation:

Crate issues

• Document what happens when log is initialized more than once
  • crate docs say “may only be initalized once”, not why, or what happens if you initialize twice
• Use env_logger instead of my_logger in example
  • env_logger is something a user can try themselves
• Crate docs should mention the logging levels and corresponding macros
• Add example for log! macro
  • using both variants
• Document max_level_*, release_max_level_* features in crate docs
• Add examples to debug!, error!, info!, trace!, warn!
  • Both pattern variants
• Expand log! docs for max_level_*
  • Seems like a non-sequiter here, but other logging macros have better explanations of this feature
• Add example for second log_enabled! pattern variant
  • There’s an example, but it only covers one variant of the macro
• LogLocation doc improvements
  • How are these values created?
  • Are these for logger implementors or log user?
  • Add an example
• LogMetadata doc improvements
  • How are these values created?
  • Are these for logger implementors or log user?
  • Add an example
• LogRecord doc improvements
  • How are these values created?
  • Are these for logger implementors or log user?
  • What’s the difference between the ‘level’/‘target’ accessors here and on the attached LogMetadata?
  • Add an example
• SetLoggerError doc improvements
  • Add links to functions
• ShutdownLeggerError doc improvements
  • Add links to functions
• LogLevel doc improvements
  • Do end-users use this type or just loggers?
  • Add link to LogLevelFilter
• LogLevelFilter doc improvements
  • Do end-users use this type or just loggers?
  • Add link to LogLevel
• max_log_level
  • Add links
• set_logger docs
  • Add links
  • Add “Errors” section
  • Add example
• set_logger_raw docs
  • Add links
  • Add “Errors” section
  • Add example
• shutdown_logger docs
  • Add links
  • Add “Errors” section
  • Add example
• shutdown_logger_raw docs
  • Add links
  • Add “Errors” section
  • Add example
• Add links to crate docs
• Add CI badges to Cargo.toml
• Change Cargo.toml ocumentation key to point to https://docs.rs/log
• Add keywords to Cargo.toml
• Change html_root_url to https://docs.rs/$crate/$version

Guidelines changes

Discussion topics

• Are there any panic conditions in this lib? Not documented.
• There are several “musts” about usage here that are kinda scary
  • should we say more about why?
  • e.g. “This function may only be called once in the lifetime of a program, and may not be called before set_logger”
    • This is a safe function and I sure can do what is says I can’t

[seanmonstar]

A question I’d add for the discussion: do all the types need the Log prefix? LogLevel, LogRecord, LogMetadata, etc? Usually we like to say that there shouldn’t be a stutter, that a crate or module name is fine for differentiation (log::LogRecord vs log::Record).

[brson]

I filled out the “future proofing” and “necessities” sections. And I discovered something that confused me really badly about the documentation on docs.rs: the README is also displayed there. I didn’t bother evaluating the README as we have no guidelines, but we need to figure out how we think README should be handled.

[brson]

@seanmonstar I added it to the discussion points.