# Flix polymorphic effects

**URL:** <https://internals.rust-lang.org/t/flix-polymorphic-effects/13395>\
**Category:** language design\
**Created:** [November 15, 2020, 9:39am UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395 "2020-11-15T09:39:30Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![leonardo](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/leonardo/32/1344_2.png) [@leonardo](https://internals.rust-lang.org/u/leonardo)\
**Post date:** [November 15, 2020, 9:39am UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/1 "2020-11-15T09:39:30Z")

</div>

(The older thread has being closed so I've written a new one).

A new language implemented in Scala for the JavaVM is named Flix: [https://flix.dev/](https://flix.dev/)

It has three interesting features, one of the three is that its type system tracks the effects of functions (I think for now it only tracks function purity, but there's interest to track other effects too later). Pure functions aren't a subtype of impure functions (invariant), so to a function that accepts an impure function as argument you can't give a pure function. The "Pure" annotation for functions is the default and you can omit it. An usage example of such polymorphic effects:

/// Assume we have some pure and impure functions: def inc1(x: Int): Int & Pure = x + 1 def inc2(x: Int): Int & Impure = Console.printLine("Hello"); x + 1

/// We can write functions that expect pure or impure functions: def twice1(f: Int -\> Int & Pure, x: Int): Int & Pure = f(f(x)) def twice2(f: Int -\> Int & Impure, x: Int): Int & Impure = f(f(x))

/// But we can also write _effect polymorphic_ functions: def twice3(f: Int -\> Int & e, x: Int): Int & e = f(f(x))

/// We can use `twice3` with both pure and impure functions: def main(): Int & Impure = twice3(inc1, 0) + twice3(inc2, 0)

The explanation of how such polymorphic effects work:

> **[oopsla2020b.pdf](https://flix.dev/paper/oopsla2020b.pdf)**
>
> 529.85 KB

The & symbol is used to attach an effect to the signature of a function. Such syntax is not beautiful, in my opinion, but it works. An example usage of such polymorphic effects to implement a map:

def map(f: a -\> b & e, xs: List [a]): List [b] & e = ...

A more complex example of polymorphic effects to implement an \>\> operator that performs function composition, it specifies that \>\> is impure if at least one of the two functions is impure (in this system Pure equals to a true boolean value, and Impure is a false value):

def \>\>(f: a -\> b & e1, g: b -\> c & e2): a -\> c & (e1 and e2) = x -\> g(f(x))

But in Flix you can do even bette, this specifies that at most one of the two input functions is impure:

def mapCompose(f: a -\> b & e1, g: b -\> c & (not e1 or e2), l: List [a]): List [c] & (e1 and e2) = ...

All this stuff is orthogonal to the current Rust type system, so I think it can be added. In practice it can't be added because it's a breaking change because on default functions must be pure. Rust used to have the "pure" annotation, but it was removed because it was implemented naively. A polymorphic effects design similar to Flix is more useful and reasonable. Still, I am not sure why/how this feature could (even in theory) improve the practice of Rust programming.

From my older thread about the same topic:

> [@\[OT\] Polymorphic effects in Flix](https://internals.rust-lang.org/t/ot-polymorphic-effects-in-flix/12363/):
>
> Flix language handles effects nicely (only purity, it seems here): [https://flix.dev/#/blog/taming-impurity-with-polymorphic-effects/](https://flix.dev/#/blog/taming-impurity-with-polymorphic-effects/) (But I don't like the effects syntax much, I think a nicer syntax could be invented. The semantics seems OK to me).

> At this point, the annotation becomes redundant,\<

I have no proof of this. I think coding in Flix for few years will answer this.

---

<div class="post-metadata">

**Author:** ![H2CO3](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/h2co3/32/2849_2.png) [@H2CO3](https://internals.rust-lang.org/u/H2CO3)\
**Post date:** [November 15, 2020, 12:20pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/2 "2020-11-15T12:20:34Z")

</div>

Alright, it sounds like it's possible, but _why_ do we want this, what is the motivation?

---

<div class="post-metadata">

**Author:** ![lordan](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/lordan/32/2107_2.png) [@lordan](https://internals.rust-lang.org/u/lordan)\
**Post date:** [November 15, 2020, 2:14pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/3 "2020-11-15T14:14:33Z")

</div>

Not sure if it's worth the effort for Rust, but it seems this could be used to distignuish between (C++) `consteval`, `constexpr` and `constinit` in a more elegant way.

Distinguishing between `Pure` and `Impure` could be used to decide whether the results of a passed-in closure can be memoized or not.

---

<div class="post-metadata">

**Author:** ![tema2](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/tema2/32/6498_2.png) [@tema2](https://internals.rust-lang.org/u/tema2)\
**Post date:** [November 15, 2020, 9:53pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/4 "2020-11-15T21:53:26Z")

</div>

There is a lot of motivation, actually. We could avoid a whole layer of problems if we had an effect system. Reasons are simple: we principally have three ways of involving side effects: using **statics** , doing **syscalls** and calling effectful functions - everything else is, actually, pure (there no another way of getting information from environment in scope of function).

Abstracting over above (e.g. do we do syscalls? do we use statics, etc.) instantly enables sound CTFE. Allowing functions to be polymorphic over both performed **syscalls** and used **statics** enables a new level of abstractions in API.

Even more can be achieved if we allow effects to include each other: `filesystem` includes `file_open`, which means that any function which has a `file_open` effect also has a `filesystem` effect, but not vice versa.

The most interesting case is data transmission via effects, as in [Effekt Language](https://effekt-lang.org), this is our `Try` (without `from` conversion, and with `!` resume type), `Generator`, `async` (which we could implement via such effects). However, all of these features have their specialized and nearly perfect designs now, so we don't have much motivation besides better APIs.

And the drawback of such system is that everything have to somehow be resumed on data effects - this turns everything into classic coroutines which raise a lot of questions.

---

<div class="post-metadata">

**Author:** ![tema2](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/tema2/32/6498_2.png) [@tema2](https://internals.rust-lang.org/u/tema2)\
**Post date:** [November 15, 2020, 10:46pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/5 "2020-11-15T22:46:30Z")

</div>

I visited previous threads. Syntax of such a feature is definitely important, also we now have backward compatibility hazards, so if it will be added someday, we should avoid requiring any bounds on existing fn's - this way we cannot enforce anything, due to that our functions currently are impure in all possible ways: they can do any syscalls, use any statics and invoke assembly. We only can invent some mechanism to forbid parts of the impurity, like "no access to statics (which are not freeze or mut)" or "no inline assembly", etc. Unsafe should allow to break this guarantees, but to certain degree.

---

<div class="post-metadata">

**Author:** ![H2CO3](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/h2co3/32/2849_2.png) [@H2CO3](https://internals.rust-lang.org/u/H2CO3)\
**Post date:** [November 16, 2020, 7:40am UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/6 "2020-11-16T07:40:04Z")

</div>

> [@tema2](#):
>
> instantly enables sound CTFE

How is that different from `const fn`, and how is `const fn` considered unsound?

> [@tema2](#):
>
> Allowing functions to be polymorphic over both performed **syscalls** and used **statics** enables a new level of abstractions in API.

Statics are only relevant when they are mutable, which they basically shouldn't ever be (I've recently seen an idea for deprecating `static mut`, because there is always a better alternative). As for syscalls, you still didn't provide a specific use case, only a very abstract argument.

Sure, abstracting over stuff can be useful. But since effects add a completely new dimension to the type system, this should not be taken lightly.

E.g. in Swift, exception handling is effectful, and that design makes life a lot difficult, for basically no return. Developers have to annotate every higher-order function with `rethrows` if they don't want to unintentionally disallow calling it with a throwing or non-throwing functional argument. At this point, it becomes redundant, and an annoying thing to remember, that proliferates through each and every functional-style API.

Doing the equivalent with syscalls would be a similarly massive pain. Want a `&mut io::Writer`? Good luck using it with `File` if the author of the API forgot to declare that it must also allow the write syscall.

---

<div class="post-metadata">

**Author:** ![atagunov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/atagunov/32/5877_2.png) [@atagunov](https://internals.rust-lang.org/u/atagunov)\
**Post date:** [November 16, 2020, 12:10pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/7 "2020-11-16T12:10:39Z")

</div>

> [@tema2](#):
>
> we principally have three ways of involving side effects: using **statics** , doing **syscalls** and calling effectful functions - everything else is, actually, pure (there no another way of getting information from environment in scope of function).

Hi there! 🙂 I no less than love your write up and I do believe effects are an extremely lucrative opportunity for development of programming languages.

However isn't this view of Rust a bit too optimistic? An `fn` using `&mut`/`Cell` to modify an argument is having an effect. Isn't it very difficult to precisely capture in a type system?

Imagine I took `fn foo() -> S {}` where `S` is a struct and did RVO manually: `fn foo(retBuf : &mut ...) {}` It's a shame this `fn` is now impure since it almost didn't change at all! I have also lost my ability to describe its behavior via it's type in sufficient detail.. Maybe `retBuf` was uninit before but is definitely init after `foo` has returned. Doing `fn foo(retBuf : &mut Option<S>)` is a waste, there's no need to pass that option tag around nor to check it after `foo` has returned.

Now imagine an `fn` modified a `struct` reachable from one of its arguments after traversing 5-6 `box` and/or `&` pointers... I wish this effect could be described by `fn` type more precisely than just "modifies something somewhere and is thus impure"..

> [@H2CO3](#):
>
> in Swift ... Developers have to annotate every higher-order function with `rethrows` if they don't want to unintentionally disallow calling it with a throwing or non-throwing functaboutional argument

Hmm... wouldn't there be a way to mitigate this somewhat at language design level? Say automatically assume `fn f(g : Fn..)` to have all the same effects as `g` _unless_ the author of `f` has used a special syntax to declare they _are not_ "passed through" from `g` to `f`? Smth like `fn f(g : Fn..) no_throw_from(g) {..` I'm talking of blocking at `fn` signature level not implementation of course.

It's true that `g` can be buried somewhere deeper within the structs passed as arguments to `f`. I'm thinking along the lines of "if the compiler can 'see' an effectful function anywhere at all in arguments of `fn foo` it automatically 'passes through' the effects, unless the author of `fn foo` has taken effort to block them in `foo`'s signature".

---

<div class="post-metadata">

**Author:** ![H2CO3](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/h2co3/32/2849_2.png) [@H2CO3](https://internals.rust-lang.org/u/H2CO3)\
**Post date:** [November 16, 2020, 12:18pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/8 "2020-11-16T12:18:28Z")

</div>

> [@atagunov](#):
>
> Hmm... wouldn't there be a way to mitigate this somewhat at language design level?

I hope so, although I haven't though much about how. The reason for that is that I've barely seen anything else in the wild, other than exception handling, that would in practice benefit from tracking effects, and at that point it feels like we're trying to solve a problem that we shouldn't even have had.

> [@atagunov](#):
>
> Say automatically assume `fn f(g : Fn..)` to have all the same effects as `g` _unless_ the author of `f` has used a special syntax to declare they _are not_ "passed through" from `g` to `f` ?

This is an interesting idea, I'd be curious how much locality the code would lose though. The immediate naive implementation would, I think at first glance, have the same problem as generic `#[derive]` has today: just because a function or function type is in the parameter list, it doesn't mean that function is going to be called at that point (just like how having a generic type parameter in the parameter list doesn't mean that the outer type contains the inner, generic one as a member).

---

<div class="post-metadata">

**Author:** ![atagunov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/atagunov/32/5877_2.png) [@atagunov](https://internals.rust-lang.org/u/atagunov)\
**Post date:** [November 16, 2020, 12:30pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/9 "2020-11-16T12:30:56Z")

</div>

> [@H2CO3](#):
>
> I'd be curious how much locality the code would lose though

I edited later.. my idea was to suppress effects "pass-through" at `fn` signature level; so code locality should not suffer.

> [@H2CO3](#):
>
> I've barely seen anything else in the wild, other than exception handling, that would in practice benefit from tracking effects, and at that point it feels like we're trying to solve a problem that we shouldn't even have had

Just to clarify you're saying `Result` is generally sufficient and exceptions can be avoided, right?

---

<div class="post-metadata">

**Author:** ![rpjohnst](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/rpjohnst/32/9524_2.png) [@rpjohnst](https://internals.rust-lang.org/u/rpjohnst)\
**Post date:** [November 16, 2020, 5:22pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/10 "2020-11-16T17:22:50Z")

</div>

> [@H2CO3](#):
>
> Statics are only relevant when they are mutable, which they basically shouldn't ever be (I've recently seen an idea for deprecating `static mut` , because there is always a better alternative).

Side note: the alternatives include `static` with interior mutability. Without that, removing `static mut` would not be a serious proposal. So this really isn't relevant to this discussion.

---

<div class="post-metadata">

**Author:** ![ratmice](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/ratmice/32/4917_2.png) [@ratmice](https://internals.rust-lang.org/u/ratmice)\
**Post date:** [November 16, 2020, 6:02pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/11 "2020-11-16T18:02:18Z")

</div>

I've seen at least a few libraries already that assume purity [GitHub - nikomatsakis/mutable: Tinkering with a more ergonomic cell abstraction](https://github.com/nikomatsakis/mutable/) and [GitHub - salsa-rs/salsa: A generic framework for on-demand, incrementalized computation. Inspired by adapton, glimmer, and rustc's query system.](https://github.com/salsa-rs/salsa)

from the README of the first,

> ## "Pure" operations
> 
> A key assumption of the library is that operations like Clone, Hash, and Eq are, in practice, "pure" -- meaning that they do not mutate any mutable cells. We can't actually **know** that this is true, however, so we enforce the constraint with a light-weight dynamic check. Thus, if you write a `Clone` impl which mutates the data in one of the types from this library (e.g., a `Mut<T>` or `MutVec<T>` ), you will get panics. Reading data from a `Mut<T>` etc should be fine though.

---

<div class="post-metadata">

**Author:** ![H2CO3](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/h2co3/32/2849_2.png) [@H2CO3](https://internals.rust-lang.org/u/H2CO3)\
**Post date:** [November 16, 2020, 7:17pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/12 "2020-11-16T19:17:20Z")

</div>

> [@atagunov](#):
>
> Just to clarify you're saying `Result` is generally sufficient and exceptions can be avoided, right?

Yes, exactly.

---

<div class="post-metadata">

**Author:** ![atagunov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/atagunov/32/5877_2.png) [@atagunov](https://internals.rust-lang.org/u/atagunov)\
**Post date:** [November 16, 2020, 7:59pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/13 "2020-11-16T19:59:45Z")

</div>

> [@H2CO3](#):
>
> Yes, exactly

Isn't Rust is a bit ironically conflicted?.. Both `Result` _and_ `catch_unwind`.. 🙂

---

<div class="post-metadata">

**Author:** ![CAD97](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/cad97/32/3460_2.png) [@CAD97](https://internals.rust-lang.org/u/CAD97)\
**Post date:** [November 16, 2020, 8:25pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/14 "2020-11-16T20:25:52Z")

</div>

> [@atagunov](#):
>
> Both `Result` _and_ `catch_unwind` .. 🙂

This talk from RustConf 2020 helps paint the difference here: [https://youtu.be/rAF8mLI0naQ](https://youtu.be/rAF8mLI0naQ)

`Result` is for any error that could reasonably ever be delt with (by something other than halting the program/thread/task and retrying). `panic!` is for unrecoverable errors, typically programmer error/invalid state, where execution cannot reasonably be continued.

`catch_unwind` isn't a way to recover from `panic!` errors. It's a way to _discard_ `panic!` errors and allowing you to try again, without taking down the hole process down with you.

---

<div class="post-metadata">

**Author:** ![atagunov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/atagunov/32/5877_2.png) [@atagunov](https://internals.rust-lang.org/u/atagunov)\
**Post date:** [November 16, 2020, 8:38pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/15 "2020-11-16T20:38:26Z")

</div>

> [@CAD97](#):
>
> `catch_unwind` ... It's ...

... and an effect handler if we have ever seen one 🙂

---

<div class="post-metadata">

**Author:** ![H2CO3](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/h2co3/32/2849_2.png) [@H2CO3](https://internals.rust-lang.org/u/H2CO3)\
**Post date:** [November 17, 2020, 1:49pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/16 "2020-11-17T13:49:42Z")

</div>

> [@atagunov](#):
>
> Isn't Rust is a bit ironically conflicted?

No. `catch_unwind` is not meant for regular error handling. It's a low-level, last resort utility for when you must absolutely catch a panic. `Result` is always preferred over panicking and catching.

---

<div class="post-metadata">

**Author:** ![atagunov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/atagunov/32/5877_2.png) [@atagunov](https://internals.rust-lang.org/u/atagunov)\
**Post date:** [November 17, 2020, 8:10pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/17 "2020-11-17T20:10:38Z")

</div>

So Flix doesn't have mutable variables? That certainly removes one effect that would be abundantly plentiful in Rust

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/system/32/14092_2.png) [@system](https://internals.rust-lang.org/u/system)\
**Post date:** [February 15, 2021, 8:10pm UTC](https://internals.rust-lang.org/t/flix-polymorphic-effects/13395/18 "2021-02-15T20:10:42Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.