Starting a Rust Mail WG

Here there!

At the RustFest Barcelona imps days, some of us met to have a chat about the mail library ecosystem in Rust. There's already a few crates that solve some of the problems one would encounter when trying to build a mail server, or any other tool that needs to handle and parse e-mail, but all of them are developed by different people and interoperability is difficult.

So we wanted to create a mail WG and we wrote a charter:


Rust Mail WG

This document outlines the basics of what it would require to start a Rust mail (e-mail) working group. Its contents should outline a charter, rationale and a roadmap.

Rationale

e-mails are cool and aren't going away (for better or worse) and we need safe mail libraries.

Charter

Goals

Rust has a lot of advantages over other languages with its memory and type safety. There are other language ecosystems that have great mail support, such as perl. There would be a real advantage to creating a safe mail ecosystem in Rust, potentially allowing many other language ecosystems to benefit from our efforts. This charter proposes to start a "Rust Mail Working Group" to facilitate this progress, and to get people who are already interested in working on mail libraries to work together on a common ecosystem.

Tasks & Activities

Create a Rust mail community

Working with mail is incredibly complex due to a series of old standards, some of which have been superseded, while old formats still need to be accepted. Many languages lack a strong mail ecosystem because of this reason. Right now the mail support in Rust is incomplete, making it hard to implement tools that work with mail in a strongly-typed manner.

One goal of this WG is to bring together people from the Rust community who are already working on mail libraries, but have so far not collaborated much to create a more solid ecosystem.

Develop an ecosystem

The primary goal isn't to aid others in the development of crates, but to be closer to the work mode of the embedded WG: it maintains and creates libraries. The mail WG aims to create a complete ecosystem of crates to do all things relevant to mail (sending, receiving, parsing, storing, loading, encrypting, decrypting, signing, and verifying signatures).

These features would be implemented in different sub-crates, which can get facaded via a central "mail" crate.

Creating a type safe, memory safe mail handling ecosystem in Rust would potentially be a huge advantage over other language ecosystems as well, which can more easily write bindings to Rust than to, say, perl.

Out of scope

To clarify, here are a few things we don't do and are not responsible for:

  • we will not implement mail applications (servers and clients)
  • we will not build semantic support for all media types, mail headers, authentication extensions etc.

Initial priorities

This section gives a summary of the topics and projects we are considering to be our initial priorities. These are topics that serve urgent, foundational needs.

Getting people involved in the WG

The first step is to advertise the working group. From talking to people at RustFest Barcelona as well as the internet it has been shown that many people are interested in solving the issues of dealing with mail.

Advertising the group, and getting interested people involved is a first important step to creating a good mail ecosystem, that is community maintained and tested, not just a few crate authors.

Implement header parsers

At a first session at RustFest it was concluded to use nom as a parser combinator framework across the ecosystem to parse mail and protocols alike. Because dealing with mail headers is difficult, starting by parsing the basic header fields into a parse-tree, then enabling other crates to do more with this tree is a good start. There are test suites in other mail libraries (namely the mail gem from Ruby) that can be ported to Rust to give us a solid foundation to build on.

Long term goals

This section gives a summary of the topics and projects we are considering to be our initial priorities. These are topics that serve urgent, foundational needs.

Create a comprehensive mail API

The "mail" crate is available to be re-used as an API facade, which would allow us to provide a unified interface for dealing with mail.

Creating a strongly-typed ecosystem

A problem with many other approaches and languages to dealing with mail is that dynamic languages (or approaches in Rust) make it harder on the developer of an application.

This would include things like a comprehensive MIME type crate, that can generate enum variants from the official IANA media types definitions or properly exposing header values as either lists or single values, without requiring the user of the mail crate to do that work instead.

First-class support for signed and encrypted mail

We don't expect every mail crate to validate and generate OpenPGP and S/MIME messages, but we would like to see first-class support for working with these types of messages. For instance, we'd like to see a MIME parser that makes it easy to work with signed OpenPGP messages even if the consumer of the message does not understand OpenPGP.


The next steps:

  1. Create an RFC to the team repo, mainly to get access to a mailing list
  2. Work on a more specific crate roadmap (which problems to solve, in which order)
  3. Put together some demo applications using the new APIs we create as a rolling test

Want to get involved? Post below or add yourself to the mailing list (coming soon)

20 Likes

Perhaps the folks over at https://github.com/async-email/ might be worth reaching out to; they've been working on a new backend in Rust for https://delta.chat/en/ (encrypted email chat app), and are just in general good humans.

4 Likes

Good point! I've pinged some of those folks on Twitter.

Yoshua Wuyts via Rust Internals rustlang@discoursemail.com writes:

Perhaps the folks over at async-email · GitHub might be worth reaching out to; they've been working on a new backend in Rust for The e-mail messenger - Delta Chat (encrypted email chat app), and are just in general good humans.

Oh, yea I had a lovely chat with a developer of Delta Chat at 36C3, funnily enough about something completely unrelated :smiley:

Anyway, here's the team PR: WIP: adding mail wg e-mail roster with no members by spacekookie · Pull Request #218 · rust-lang/team · GitHub

I don't know who's part of the teams repo yet, which is rather time consuming so I just added everybody to the mail roster for now, with no actual members. Also, I think a WG leadership position to organise meetings and whatever would be good, and it'd be even better if that could be someone else than me :))

Might be worth reaching out to the author of the meli MUA too. They’re https://chaos.social/@epilys on the fediverse.

1 Like

Thanks for starting this and thinking of us. Definitely interested in getting involved and growing the ecosystem!

yes, thanks for thinking of us -- i admit i am not sure what a Rust WG entails yet, but happy to find out :slight_smile:

I've also pinged the author of a decent mimeparser that Delta Chat is using: https://github.com/staktrace/mailparse/issues/54

Thanks @hpk42 for the ping (and also to @djc for the twitter ping a while back which I only just saw.. sorry about that, I don't use twitter much).

As the author of the mailparse crate I'm interested in this discussion and am happy to provide input if that would be helpful. I'm not sure exactly what is entailed in joining the WG; if it requires a large time commitment (F2F meetings etc) then I'm probably not interested but if it's just async communication then I'm in.

1 Like

We have a number of mail-related closed source libraries (mime parser, mta server and client) that we would like to open source and join this WG. We are planning to invest a lot into email related rust infrastructure in the future.

3 Likes

I'm the author of the lettre crate, and I'm definitely interested in joining a mail WG!

1 Like

Hy, I'm kinda the author of the mail crate (not code I'm proud of as I have learned many thinks since then. But not useless code either).

Anyway I got confirmation from DAC GmbH that I can pass on the mail/mail-* and related crate names to the Mail WG working group to prevent it from becoming a Zombie crate (the code is anyway under MIT/Apache-2; DAC GmbH due to organizational changes won't maintain/complete that crates).

This is mainly mail, mail-*, new-tokio-smtp (the last one is a tokio SMTP client crate, not feature complete but usable for sending mails over SMTP, also it was written during a time where tokio re-structured some points, so it needs a a bit of rewriting to use some usefull tokio tooling and async).

So count me in, too :+1:

5 Likes

Hey, I have zero experience with any mail related stuff but would like to join as well if it’s okay :+1:

My experience at poking at various mail parsing libraries has been that every single one has been deficient in some way or another, as email is very much a situation where people will ignore the specifications and throw gibberish at you and expect you to make sense of it. One of the chief offenders is charsets; I've got a fair amount of email that's using non-UTF-8 8-bit headers (although perhaps that's diminished in the past several years, as people finally wake up to UTF-8). RFC 2047 is another area of pain; I've got data literally showing that it's more often than not implemented incorrectly.

As a result of these experiences, I've soured on general-purpose libraries in this domain, since far too frequently, these libraries try to cover up the pain of email with too much magic, but fail to actually provide enough knobs to let you do the right thing if you need to handle the poorly-encoded emails.

In the hope of being persuaded otherwise, I have some interest and experience here.

3 Likes

Yes, encoded words are a total annoyance. My favorite is the inter-working of the length limit with the fact that a single WS between two encoded words must be ignored. Which means you can not split by words and then encode each word, but neither can you encode everything as one encoded word. And to top that of multiple mail client's have their own non standard way of encoding white spaces between encoded words :sob:

Oh and you have to make sure the raw input data doesn't happen to contain something which could be considered as the start of an encoded word and if you need to encode that, too.

Then you can mix multiple encodings even in the same header.

Not speaking about the fact that the "content" of an encoded word can be base64 or quoted printable but depending on the context where it appears there are different constraints which characters need to be encoded, so it's not the same quoted printable encoding as if it appears in a mail body :cry:

To just name a view problems with it.

1 Like

any news here? are there other communication channels that i'm missing? interested in this topic as well, but not sure where to start :slight_smile:

Hey there,

any news here? are there other communication channels that i'm missing? interested in this topic as well, but not sure where to start :slight_smile:

Not really unfortunately. There was a lot of momentum after Rust Fest and we had an e-mail chain going where we talked about setting up a group charter, what would be in-scope and what would be out of scope. But at some point the engagement kinda fizzled out. I think everybody was already busy with all their other stuff, and then the pandemic happened, etc.

Bottom line is that I'm not aware of any progress that's been made. I would love to actually get a working group for mail stuff started, but I'm also aware that my time is very limited, and it would have to have some other than me leading it x)

How is the mailing list business going on? I'm interested in joining if there is one.

Hey! I haven't really been checking the forum so sorry for missing this. The WG never really formed. We developed a charter, but when it became time to find a WG leader for the beginning (and me realising I couldn't do it because of time), nobody else stepped up and the project just kinda died.

Not really sure what it would take to resurrect the WG idea. Generally people seemed interested in hacking on things but weren't really able or willing to collaborate much on "big picture" planning.

I also want to join, I have enough time to participate in it, hope to work with you guys.:nerd_face::nerd_face: