# Synchronized FFI access to POSIX environment variable functions

**URL:** <https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475>\
**Category:** Unsafe Code Guidelines\
**Created:** [October 21, 2021, 2:35pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475 "2021-10-21T14:35:49Z")\
**Posts on this page:** 20\
**Page:** 13

<div class="post-metadata">

**Author:** ![atagunov](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/atagunov/32/5877_2.png) [@atagunov](https://internals.rust-lang.org/u/atagunov)\
**Post date:** [January 7, 2022, 5:38pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/241 "2022-01-07T17:38:51Z")

</div>

> [@RalfJung](#):
>
> that past but this thread is about moving away from it

...an alternative point of view may be that this thread [is](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/178)[about](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/196) convincing libc maintainers to introduce a thread-safe API for both reading and writing the environment.

Once they do - in 10 years from now - Python, Rust, C, ... will be able to happily co-exist in the same process. Even if due to circumstances out of your control by the time you get a chance to set environment some other threads are already running.

I also got an impression that to a lesser extent this thread was about providing some stop-gap measures that will allow Rust programmers/programs to desperately hold onto safety by fingernails until that happens.

P.S. honestly in my view libc/posix maintainers deserve little credit for not having introduced such API-s tens of years ago..

---

<div class="post-metadata">

**Author:** ![RalfJung](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/ralfjung/32/2415_2.png) [@RalfJung](https://internals.rust-lang.org/u/RalfJung)\
**Post date:** [January 7, 2022, 9:40pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/242 "2022-01-07T21:40:48Z")

</div>

True, in some glorious future we will hopefully have a safely mutable global environment. There's clearly a need for that. Sadly, that will take a while.

But in the present time, it seems pretty clear that the environment is effectively read-only (with some exceptions). As such, it makes sense to treat it like that in Rust: reading is safe, and writing is `unsafe` and exists for those exceptions. This is different from `static mut`. (It is somewhat comparable to a library with an internal `static mut` that exposes a safe getter and an unsafe setter and documents when the setter is safe to use -- that would be a totally reasonable abstraction to have.)

---

<div class="post-metadata">

**Author:** ![safinaskar](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/safinaskar/32/8969_2.png) [@safinaskar](https://internals.rust-lang.org/u/safinaskar)\
**Post date:** [January 28, 2022, 1:45am UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/243 "2022-01-28T01:45:37Z")

</div>

> [@zackw](#):
>
> Exposure of internal C library lock objects is unprecedented

This is not true. "ftrylockfile" from glibc on Linux (I don't know whether it is standardized somewhere) is exactly such lock. As well as I understand it is exactly lock internally held by "printf"

---

<div class="post-metadata">

**Author:** ![xfix](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/xfix/32/2726_2.png) [@xfix](https://internals.rust-lang.org/u/xfix)\
**Post date:** [January 28, 2022, 8:15am UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/244 "2022-01-28T08:15:23Z")

</div>

It is a POSIX API actually.

---

<div class="post-metadata">

**Author:** ![zackw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/zackw/32/2071_2.png) [@zackw](https://internals.rust-lang.org/u/zackw)\
**Post date:** [January 28, 2022, 7:01pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/245 "2022-01-28T19:01:54Z")

</div>

> [@safinaskar](#):
>
> > [@zackw](#):
> >
> > Exposure of internal C library lock objects is unprecedented
> 
> This is not true. "ftrylockfile" is exactly such lock

That's really not the same thing. The (POSIX) API lets you lock individual FILEs, but the _lock object_ embedded in each FILE is not exposed — you don't have any way of knowing whether it's a `pthread_mutex_t` or a `pthread_rwlock_t` or a C2011 `mtx_t` or what, and you don't get as much manual control over the lock as you would if you did. This means the library has more flexibility in how it implements the locking.

---

<div class="post-metadata">

**Author:** ![Kixunil](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kixunil/32/4110_2.png) [@Kixunil](https://internals.rust-lang.org/u/Kixunil)\
**Post date:** [February 11, 2022, 3:55pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/246 "2022-02-11T15:55:09Z")

</div>

I came up with a potential solution to this I didn't see proposed yet (many comments?) and did some initial work to determine its feasibility. The idea is to just fork the affected library (localtime but could be repeated for others) and patch fetching of env vars to use Rust code. This is similar in effect to RIIR except it's much easier.

Here's my library: [GitHub - Kixunil/rl\_localtime: Rust-locked localtime - a sound localtime implementation](https://github.com/Kixunil/rl_localtime)

Please take a look if you're interested in fixing `chrono`. Note that I've also posted this to the `chrono` issue.

---

<div class="post-metadata">

**Author:** ![jhpratt](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jhpratt/32/11640_2.png) [@jhpratt](https://internals.rust-lang.org/u/jhpratt)\
**Post date:** [February 11, 2022, 10:21pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/247 "2022-02-11T22:21:41Z")

</div>

I haven't looked into this much, but keep in mind that the core issue is that `std::env::set_var` is simply unsound. Theoretically I, as the author of the time crate, shouldn't have to worry about `localtime_r` leading to a data race because `set_var` shouldn't ever be called when multiple threads are running.

---

<div class="post-metadata">

**Author:** ![tcsc](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/tcsc/32/3806_2.png) [@tcsc](https://internals.rust-lang.org/u/tcsc)\
**Post date:** [March 2, 2022, 9:24pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/248 "2022-03-02T21:24:03Z")

</div>

So, this has largely fallen off the thread, and while people were broadly in agreement that something needs to be done here (and we even had some agreement on some of the steps that need to be taken, although certainly not all), it's not 100% what next steps here would be.

I would... quite prefer this not be completely forgotten though.

---

<div class="post-metadata">

**Author:** ![josh](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/josh/32/5934_2.png) [@josh](https://internals.rust-lang.org/u/josh)\
**Post date:** [March 2, 2022, 9:47pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/249 "2022-03-02T21:47:27Z")

</div>

I don't think people have forgotten about this, it just needs steps that aren't happening in this thread.

@zackw is driving the long-term plan of making safe environment functions. That'll take a long time before we can use it, but it seems like the only path towards a future safe interface.

I've seen a couple of different efforts towards making aspects of Rust libraries easily configurable by means other than environment variables, to reduce the need for `set_var` in Rust code.

And there's a proposal in the works to make unsafe environment functions and possibly deprecate the safe ones.

---

<div class="post-metadata">

**Author:** ![zackw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/zackw/32/2071_2.png) [@zackw](https://internals.rust-lang.org/u/zackw)\
**Post date:** [March 2, 2022, 10:03pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/250 "2022-03-02T22:03:18Z")

</div>

Just want to confirm that I haven't forgotten about the long-term plan. My day job has been keeping me very busy, but when I have something to report I will post it here (probably in a new thread).

---

<div class="post-metadata">

**Author:** ![the8472](https://avatars.discourse-cdn.com/v4/letter/t/0ea827/32.png) [@the8472](https://internals.rust-lang.org/u/the8472)\
**Post date:** [July 31, 2022, 4:55pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/251 "2022-07-31T16:55:53Z")

</div>

On linux that would be fairly cheap via `unshare` with `CLONE_VM` which either has no effect (single-threaded) or returns an error (multi-threaded).

```rust
       In addition, CLONE_THREAD, CLONE_SIGHAND, and CLONE_VM can be
       specified in flags if the caller is single threaded (i.e., it is
       not sharing its address space with another process or thread).
       In this case, these flags have no effect. (Note also that
       specifying CLONE_THREAD automatically implies CLONE_VM, and
       specifying CLONE_VM automatically implies CLONE_SIGHAND.) If the
       process is multithreaded, then the use of these flags results in
       an error.

```

---

<div class="post-metadata">

**Author:** ![bascule](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/bascule/32/3057_2.png) [@bascule](https://internals.rust-lang.org/u/bascule)\
**Post date:** [July 31, 2022, 9:13pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/252 "2022-07-31T21:13:52Z")

</div>

> [@josh](#):
>
> And there's a proposal in the works to make unsafe environment functions and possibly deprecate the safe ones.

Exciting! I assume it will be posted here whenever it's online?

---

<div class="post-metadata">

**Author:** ![jhpratt](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jhpratt/32/11640_2.png) [@jhpratt](https://internals.rust-lang.org/u/jhpratt)\
**Post date:** [July 31, 2022, 9:17pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/253 "2022-07-31T21:17:49Z")

</div>

I've had a PR open for a while introducing the new unsafe functions. It's an "either or" situation, though. We either want the new methods _or_ mark the existing ones as "deprecated safe", which to my knowledge is still WIP.

---

<div class="post-metadata">

**Author:** ![Kixunil](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kixunil/32/4110_2.png) [@Kixunil](https://internals.rust-lang.org/u/Kixunil)\
**Post date:** [August 2, 2022, 5:01pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/254 "2022-08-02T17:01:06Z")

</div>

> [@jhpratt](#):
>
> `std::env::set_var` is simply unsound

I don't believe so. If you write pure Rust without `unsafe` it can't cause UB. But it being `safe` makes writing sound FFI code impractical. FFI code being impractical to write seems to be a good reason to make it `unsafe` though.

---

<div class="post-metadata">

**Author:** ![zackw](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/zackw/32/2071_2.png) [@zackw](https://internals.rust-lang.org/u/zackw)\
**Post date:** [August 2, 2022, 6:34pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/255 "2022-08-02T18:34:38Z")

</div>

> [@Kixunil](#):
>
> > [@jhpratt](#):
> >
> > `std::env::set_var` is simply unsound
> 
> I don't believe so. If you write pure Rust without `unsafe` it can't cause UB

Rust's own locking _probably_ rules out UB, on Unix, as long as there's no FFI, but I'm not 100% sure; anyway, the risk is if another thread is in the middle of `std::env::var()` for the same variable, has exited the internal call to `libc::getenv` and is in the process of copying the value into a `String`, there could be a data race between that thread and `set_var` → `libc::setenv` → `free` on the memory block holding the old contents of the variable. Given the way `std` is set up, I _think_ this would be a plain old bug in the library as long as there's no FFI.

---

<div class="post-metadata">

**Author:** ![jhpratt](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jhpratt/32/11640_2.png) [@jhpratt](https://internals.rust-lang.org/u/jhpratt)\
**Post date:** [August 2, 2022, 7:49pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/256 "2022-08-02T19:49:39Z")

</div>

> [@Kixunil](#):
>
> If you write pure Rust without `unsafe` it can't cause UB

> [@zackw](#):
>
> Rust's own locking _probably_ rules out UB, on Unix, as long as there's no FFI, but I'm not 100% sure

Unfortunately this has been demonstrated to not be the case. Pure Rust code can segfault. [https://github.com/rust-lang/rust/issues/27970#issuecomment-965568295](https://github.com/rust-lang/rust/issues/27970#issuecomment-965568295)

Even if this weren't the case, `std::env::set_var` is still unsound, as _getting_ an environment variable is thread-safe while _setting_ it is not. By not using the same lock everywhere (which is currently impossible), there is no way to synchronize gets and sets when using FFI. This is why I had to resort to the `num_threads` crate — if multiple threads exist, there's no way to guarantee anything.

---

<div class="post-metadata">

**Author:** ![Kixunil](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/kixunil/32/4110_2.png) [@Kixunil](https://internals.rust-lang.org/u/Kixunil)\
**Post date:** [August 3, 2022, 9:26am UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/257 "2022-08-03T09:26:14Z")

</div>

That's an internal library bug then that's fixable by changing non-public code in the library. Contrast this with things like unsoundness in `Pin` which couldn't be resolved without breaking change.

---

<div class="post-metadata">

**Author:** ![jhpratt](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/jhpratt/32/11640_2.png) [@jhpratt](https://internals.rust-lang.org/u/jhpratt)\
**Post date:** [August 3, 2022, 7:40pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/258 "2022-08-03T19:40:27Z")

</div>

Conceivably, yes. But it's still unsound because Rust provides no way to use the same lock when performing FFI. Said another way, there is no way for me to call `getenv` or an equivalent method in FFI without causing a race condition. This is despite `getenv` explicitly being thread-safe. As such the Rust method setting the environment is unsound, as that is explicitly _not_ thread safe.

---

<div class="post-metadata">

**Author:** ![RalfJung](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/ralfjung/32/2415_2.png) [@RalfJung](https://internals.rust-lang.org/u/RalfJung)\
**Post date:** [August 3, 2022, 7:44pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/259 "2022-08-03T19:44:40Z")

</div>

> [@Kixunil](#):
>
> I don't believe so. If you write pure Rust without `unsafe` it can't cause UB.

That's why I still think we should have `env::{rust_var,set_rust_var,remove_rust_var}` which modifies a Rust-only 'shadow' environment (also honored by `Command`). This will allow easy porting of existing sound Rust code that does not want or need to interact with the underlying system environment of the current process (but might still want to properly set up the system environment of new processes it spawns).

In an ideal world we wouldn't have such a shadow environment. If I we could un-do having these safe methods in Rust 1.0 we should do it. But we cannot, and given the current situation at hand, I think we do need to give programmers something to migrate to, or this will be a disaster transition.

> [@jhpratt](#):
>
> I've had a PR open for a while introducing the new unsafe functions. It's an "either or" situation, though. We either want the new methods _or_ mark the existing ones as "deprecated safe", which to my knowledge is still WIP.

I feel like we probably need an RFC to hash out and decide about the plan here. Just local discussions in a PR won't do, precisely because there are multiple alternatives.

---

<div class="post-metadata">

**Author:** ![josh](https://sea2.discourse-cdn.com/flex002/user_avatar/internals.rust-lang.org/josh/32/5934_2.png) [@josh](https://internals.rust-lang.org/u/josh)\
**Post date:** [August 3, 2022, 7:57pm UTC](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475/260 "2022-08-03T19:57:47Z")

</div>

I really don't think we should offer a "shadow environment"; that seems both like a trap for users (often not what they're actually looking for) and a suboptimal interface for things that are exclusively Rust. Working with `Command` but not the current process will work in some cases while surprising people in others, and it would also add complexity (what happens if the shadow environment and the real environment conflict).

As soon as `#[deprecated_safe]` is available, I think we should use that on the existing functions, and continue having that be our interface to the environment.

[Previous page](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475.md?page=12)

[Next page](https://internals.rust-lang.org/t/synchronized-ffi-access-to-posix-environment-variable-functions/15475.md?page=14)
