Routing and extraction in Tide: a first sketch

[Nemo157]

Shouldn’t rustc be able to tell that the two implementations are for disjoint sets of types? i.e. that a function/closure with one parameter can never be the same type as a function/closure with zero parameters…

If you manually implement the Fn trait it’s entirely possible to have a single callable take multiple different argument sets (playground example). Still unstable (and a real pain to actually invoke), but the current trait definitions allow for it.

[Keats]

I like what’s in the post and think waiting for some improvements to Rust for a better UX is totally worth it.

Something that seems missing (or that I didn’t see) is a way to resolve (called reverse in Django) a URL in the views.

Let’s say you have the following routes:

order-details: /orders/{id}
checkout: /checkout

and imagine you want to redirect the user after a POST on the checkout route to its order page with the newly created order ID. In Django you would redirect to the URL given by reverse("order-details", kwargs={'id': order.id}).

This requires 2 things (well 3, I’ll expand on the last one a bit later):

• named routes: if I change a URL, I only want to change it at one place, not everywhere
• named parameters: we could potentially skip that but it avoids tons of bugs and improve readability so that would be sad to not have them

The last point that would be nice, providing we have named routes is namespaces. It has been mentioned before as subrouters providing some routes.

If I do router.mount("/auth", "auth", &auth_urls) with the parameters being (prefix, namespace, router), I should be able to redirect to /auth/login by doing reverse("auth:login") for example if we follow the Django example and there is a view named login in the auth subrouter.

[dcarosone]

I like what’s in the post and think waiting for some improvements to Rust for a better UX is totally worth it.

I think it goes both ways: this can be a showcase for what can be done with the new features, too.

[efindlay]

Addressing @aturon’s original post, I think looking at routing and extraction at this early point in the development process is too narrow.

Security, consistency and reliability need to be the guiding considerations in any networking software. If we had these components in a web framework it would take web programming to another level. On the other hand, without them Rust’s value is diminished. Why lean on Rust’s memory-safety if our network software isn’t secure? Why lean on Rust’s reliablity if our network software isn’t fault tolerant? Why prevent data-race inconsistencies if our distributed data isn’t consistent?

Consider the example of a web server that is a very basic digital bank. Lets say it returns a bank balance or debits one account by a u64 while at the same time crediting another account with the same amount. Hopefully, this should be easy to build but unfortunately, without solid security, consistency and reliability support it becomes far from straightforward.

One of Rocket’s good points is that it makes input validation and authorization explicit. One of Actix’s good points is that the Actor model is potentially a good solution for reliability and consistency. I’d like to see something along the lines of the banking server example above as one test of a successful outcome to the Tide project.

[hjr3]

It seems like annotations go against the design goals of plain rust. To me, an annotation and a macro are not that different for the user.

[ahmedcharles]

I mostly agree, but a solution that doesn’t use macros/attributes seems to have the property that path captures are positional and it’s not obvious how to track the various positions from one endpoint argument to the next. The Request would have to track how many times it was asked for a path capture and that would require that it be &mut, which isn’t really that great.

[nicoburns]

If you manually implement the Fn trait it’s entirely possible to have a single callable take multiple different argument sets (playground example ). Still unstable (and a real pain to actually invoke), but the current trait definitions allow for it.

Oh huh, so you can very nearly do function overloading in nightly rust. That would be awesome if it ever gets stabilised (and potentially remove the need for a lot of uses of specialization?)

Riffing further on this: we could do away with Path<T> entirely, in favor of directly deriving an Extractor impl:

#[derive(Extractor)]
#[extractor(path, component = "user_id")]
struct UserId(u64);

which gets rid of the annoying need for .0 to strip away the Path wrapper. (This of course assumes that you want to use the UserId wrapper type rather than immediately working with the raw u64 ).

Hmm… I think extractors are much more useful if they’re composable with types, rather than tied to them. Whose to say you don’t want to extract the UserId from the path in one route, and from the query string in another (maybe not the best API design, but these things happen)?

On a tangentially related note, it has occurred to me that it would be super nice for these kinds of api’s if you could have associated types on Structs and use User::Id in place of UserId (would save importing an extra type, and would be less confusing if for example the UserId type was autogenerated by a “model” macro…

[Yato]

Why does (rust’s type system work such that) the Endpoint trait requires the Kind parameter? I came across this when trying to implement rustdi too. In the following case:

impl<T, Ret> Foo for T where T: Fn() -> Ret { ... }

and

impl<T, Ret, T0> Foo for T where T: Fn(T0) -> Ret { ... }

Shouldn’t rustc be able to tell that the two implementations are for disjoint sets of types? i.e. that a function/closure with one parameter can never be the same type as a function/closure with zero parameters…

But those two are not disjoint, you could have a user defined type that implements both the 0 arg Fn and the 1 arg Fn. Yes this probably won’t happen, but it is possible. Fn is just a trait.

[aturon]

Logistical note: I’m on PTO this week (of Oct 22nd), so won’t be able to follow up more until next week. At that point I’m hoping we can get the repository in shape for collaboration and break up some of these conversation into issues!

[nicoburns]

New thought. Could we define the extractor trait with an input type like:

trait Extractor<Input> {
    type Output;
    fn extract(&self, args: Input) -> Self::Output;
}

Then say tide defines all it’s extractors as Extractor<Request, AppState, Config> and blanket impls for subsets of this (Extractor<Request, Config>, Extractor<Request, Config>, etc). Then extractor implementations could depend on what they actually need, and could be reused with any framework that provides those things?

It’s a shame there’s no way to treat type parameters as unordered…

[kamalmarhubi]

One thing I haven’t seen captured here or in any of the other frameworks: the idea of a request lifetime. The most immediate application would be to allow borrowing from headers and the path / query string, but it could extend to borrowing from the body for small JSON bodies, depending on the body’s type. Avoiding allocation and copying here feels like it would be a win.

Further down the line, it would be interesting to play with request scoped allocators. I haven’t been keeping up with the allocator-generic collections APIs, but it would be fun to see something like Vec::with_capacity_in(32 * 1024, context.allocator()), and providing a simple bump allocator that does nothing on dealloc. Once the request is done, the whole chunk of memory can be freed—likely to a pool of such chunks to be used for other requests.

[aturon]

Once the request is done, the whole chunk of memory can be freed—likely to a pool of such chunks to be used for other requests.

Wow, what a cool idea! It seems quite feasible to me.

Right now due to compiler limitations it’s not possible to work with async fn definitions that are lifetime-parametric. Once that limitation is lifted, it should be feasible to do what you’re suggesting here.

[aturon]

I’ve got two exciting updates!

• I’ve put up an initial implementation of Tide, and filed 25+ issues for folks who want to dig in.
• I’ve published a blog post talking about a middleware design, and the related idea of “computed values”. Check it out!

I’ll comb through this thread again a bit later this week and try to open some additional issues for further discussion, and link them here.

[scottmcm]

One thing I haven’t seen captured here or in any of the other frameworks: the idea of a request lifetime.

I really like that thought. AspNetCore, for example, has request-scoped things in their DI container, but of course there’s no checking and many times I’ve seen bugs from people passing something resolved to something static-scoped that ends up getting cached across requests causing weird problems at runtime. Having an actual lifetime for this sounds great – it reminds me of the 'tcx in rustc…

(Edit: there could even be a method like Box::leak that returns things 'request instead of 'static.)

[Nemo157]

Couple of questions related to Compute:

Why does it need mutable access to the request? This seems like it could potentially cause ordering issues if there’s multiple computed values that modify the same parts of the request, depending on what order you request their values they may interfere with each other.

Is there a reason this necessarily has to be a custom trait? First thought would be to just use Future, and keep an &Request in the future’s context, although this would require pinning the request somewhere. Alternatively maybe an async closure like for<'a> FnOnce(&'a Request) -> impl Future could be possible (which would then be hidden inside the request somehow and memoized to allow acquiring it multiple times while only running the operation once).

[nicoburns]

Re: The proposed middleware API. I feel like it shouldn’t be a Result in Result<Request, Response>. As there may also be use cases for a middleware returning a Response where that doesn’t indicate an error condition (CORS middleware responding to OPTIONS requests for example). Maybe a custom enum would make sense here. Although in fact it would be a perfect case for the newly proposed anonymous/sum enums.

[kamalmarhubi]

Right now due to compiler limitations it’s not possible to work with async fn definitions that are lifetime-parametric. Once that limitation is lifted, it should be feasible to do what you’re suggesting here.

Ah that’s unfortunate! Is this a case of not-implemented, or not-specified? Is there a tracking issue?

[kamalmarhubi]

I feel like it shouldn’t be a Result in Result<Request, Response> . As there may also be use cases for a middleware returning a Response where that doesn’t indicate an error […]

This sounds right to me. Perhaps something like

pub enum MiddlewareResult {
  Continue(Request),
  Respond(Response),
}

Bikesheddable abbreviations: Cont and Resp.

Edit: or a possibly clearer

pub enum MiddlewareResult {
  Continue,
  Respond(Response),
}

together with changing the methods to take &mut Request. If the middleware wants to use a completely unrelated request, it can use mem::replace.

[kamalmarhubi]

While boxing the futures has some performance cost, it’s expected that the cost is extremely minimal, and boxing allows us to avoid much more complicated type tracking (and associated lengthy compile times).

@aturon can you expand on this a tiny bit? I get the compile time issue—though I wonder if you think it’ll be problematic in typical middleware stacks—but what do you mean by extra type tracking?

[aturon]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.