# Libsystem, or, the great libstd refactor
**URL:** https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765
**Category:** libs
**Created:** [October 9, 2015, 5:51pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765 "2015-10-09T17:51:40Z")
**Posts on this page:** 18
**Page:** 2
### Author:  [@briansmith](https://internals.rust-lang.org/u/briansmith)
#### Post date: [January 12, 2016, 4:00am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/21 "2016-01-12T04:00:44Z")
> [@arcnmx](#):
>
> Well his more recent comment was pretty dismissive so it seems unlikely that's a good approach to take... I'm not so much looking for approval as much as indication from someone that I'm not doing something very wrong here, but no one seems to care enough to look.
Nobody has time to read a 166 file +3342/-3030 patch. If you want somebody to look at a patch, you need to make one that is very much smaller. If you won't break up the patch into smaller parts until the Rust community agrees that they want to split all the platform-dependent bits of libstd into separate modules, then the way to get that kind of agreement is through an RFC.
[quote] As for an RFC... "cut-paste anything in libstd within an adhoc conditional #[config(target\_os = ...)] to the sys module" doesn't seem like a very productive one to me so meh. [/quote]
Probably more people would be supportive of that wording than the original way it was proposed. đ
But, your proposal is more than that, right? I've made an attempt to help you make the RFC here: [https://gist.github.com/briansmith/d4540b86c113eba8ee93](https://gist.github.com/briansmith/d4540b86c113eba8ee93). I intentionally left out the technical details in the "Detailed Design" section as you know them better than us. Please fill them in and make other changes for any other parts of your proposal that I've misunderstood.
---
### Author:  [@arcnmx](https://internals.rust-lang.org/u/arcnmx)
#### Post date: [January 12, 2016, 4:43am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/22 "2016-01-12T04:43:05Z")
> [@briansmith](#):
>
> Nobody has time to read a 166 file +3342/-3030 patch. If you want somebody to look at a patch, you need to make one that is very much smaller.
Eh one can glance it over and look at the file structure and an example module or two ;\_;
> [@briansmith](#):
>
> I've made an attempt to help you make the RFC here
You are good at this communication thing. The "formal verification" part is a bit hard to define though. We're not really defining the standard library so much as the layer underneath it that's used to interact with the underlying OS. And even that definition isn't really that formalized.
---
### Author:  [@briansmith](https://internals.rust-lang.org/u/briansmith)
#### Post date: [January 12, 2016, 5:22am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/23 "2016-01-12T05:22:44Z")
> [@arcnmx](#):
>
> Eh one can glance it over and look at the file structure and an example module or two ;\_;
That's generally not how people work, for better or worse.
> [@arcnmx](#):
>
> The "formal verification" part is a bit hard to define though. We're not really defining the standard library so much as the layer underneath it that's used to interact with the underlying OS. And even that definition isn't really that formalized.
I agree I wasn't so clear, but I'm not sure how to improve it. The point I was trying to make is that having a clear interface to the platform-specific bits should help the development of a formal definition of the Rust standard library by facilitating the removal of assumptions that depend on the semantics of libc...eventually.
---
### Author:  [@oli-obk](https://internals.rust-lang.org/u/oli-obk)
#### Post date: [January 12, 2016, 8:27am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/24 "2016-01-12T08:27:25Z")
How about doing this incrementally?
Add a libsystem, but only port a single thing. Then port the next thing in another PR. At some point libstd should be free of system dependent code, and libsystem should only offer the minimal public API necessary for libstd to work.
---
### Author:  [@Zoxc](https://internals.rust-lang.org/u/Zoxc)
#### Post date: [January 15, 2016, 6:45am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/25 "2016-01-15T06:45:52Z")
As someone who wants to add support for another OS to libstd I like the idea. Keeping the OS specific code separated also makes it easier to maintain a fork which supports an additional OS.
---
### Author:  [@lilred](https://internals.rust-lang.org/u/lilred)
#### Post date: [February 16, 2016, 4:26pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/26 "2016-02-16T16:26:31Z")
+1, Iâd love for this to happen.
Itâd be really neat if `libstd` could be implemented inside Rust code the same way `libcore` is - by just providing x,y,z language items.
---
### Author:  [@briansmith](https://internals.rust-lang.org/u/briansmith)
#### Post date: [February 16, 2016, 7:10pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/27 "2016-02-16T19:10:48Z")
See this Hacker News thread: [https://news.ycombinator.com/item?id=11111059](https://news.ycombinator.com/item?id=11111059)
Rust programs vulnerable to buffer overflow because of glibc.
@arcnmx, Are you going to be able to fill out the technical details of the RFC? It doesnât have to be perfect. Just an outline of the various refactorings you did, before you completely forget what they are.
---
### Author:  [@arcnmx](https://internals.rust-lang.org/u/arcnmx)
#### Post date: [February 16, 2016, 7:24pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/28 "2016-02-16T19:24:31Z")
Yeah, sorry, Iâve been busy⌠I will try to get to that asap so at least we have some talking points and can get to it all sooner rather than later.
---
### Author:  [@steveklabnik](https://internals.rust-lang.org/u/steveklabnik)
#### Post date: [February 16, 2016, 7:40pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/29 "2016-02-16T19:40:16Z")
> [@briansmith](#):
>
> Rust programs vulnerable to buffer overflow because of glibc.
Unless you're using a MUSL-related rustc. Which is admittedly a minority.
---
### Author:  [@arcnmx](https://internals.rust-lang.org/u/arcnmx)
#### Post date: [February 16, 2016, 11:39pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/30 "2016-02-16T23:39:12Z")
>
---
### Author:  [@tomjakubowski](https://internals.rust-lang.org/u/tomjakubowski)
#### Post date: [February 17, 2016, 1:50am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/31 "2016-02-17T01:50:36Z")
Or, almost definitely, if youâre running Rust on OS X or Windows. đ
---
### Author:  [@notriddle](https://internals.rust-lang.org/u/notriddle)
#### Post date: [February 18, 2016, 5:38am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/32 "2016-02-18T05:38:32Z")
I thought the MinGW toolchain used glibcâŚ
---
### Author:  [@retep998](https://internals.rust-lang.org/u/retep998)
#### Post date: [February 18, 2016, 5:52am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/33 "2016-02-18T05:52:17Z")
> [@notriddle](#):
>
> I thought the MinGW toolchain used glibc...
No, MinGW uses MSVCRT still, it just uses the really _really_ old one in `msvcrt.dll` that you're not supposed to use, and because that `msvcrt.dll` is missing a significant amount of C stuff, MinGW implements the missing functionality on top of it. To use glibc on Windows would require a massive overhaul of it to use windows API instead of linux syscalls.
Furthermore Rust itself uses very little of the C standard library, just `strlen` and various math functions along with `memcpy` and friends. Everything else Rust does directly through Windows API.
---
### Author:  [@eddyb](https://internals.rust-lang.org/u/eddyb)
#### Post date: [February 18, 2016, 6:32pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/34 "2016-02-18T18:32:18Z")
I am surprised nobody mentioned allocators!
Most of the libstd code that doesnât depend on POSIX-compatible I/O and multithreading is not in libcore due to it necessitating dynamic memory allocation.
We already have `libcollections` (missing `HashMap` pending defaulting in inference), and with custom allocator support, they wouldnât even depend on `liballoc`.
`libstd` would then contain re-exports, plugging its default allocator as the default for all the collections/smart pointers, and anyone could do the same, skipping all the extra OS-dependent features `libstd` has.
AFAIK that was the plan, although @nikomatsakis seems to suggest that the necessary âdefaulting to type parameter defaults in inferenceâ mechanism might never be stabilized.
---
### Author:  [@Amanieu](https://internals.rust-lang.org/u/Amanieu)
#### Post date: [February 18, 2016, 10:19pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/35 "2016-02-18T22:19:14Z")
Another possibility for allowing code that requires memory allocation to be used in kernel/embedded environments would be to add a `none` system backend to libstd in addition to the existing `windows` and `unix`. This backend would simply return an error for all OS-specific functionality to indicate that it is not implemented. This is similar to what the [newlib C library does](https://sourceware.org/newlib/libc.html#Syscalls) to support running on bare metal systems.
---
### Author:  [@briansmith](https://internals.rust-lang.org/u/briansmith)
#### Post date: [February 18, 2016, 10:37pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/36 "2016-02-18T22:37:48Z")
Eddy, Amanieu, I think that it is also important to figure out how to get more of the standard library to work in environments where allocation needs to be different than the typical `malloc`/`free` style heap. But, I think thatâs an orthogonal or otherwise separate enough to whatâs being proposed here to have itâs own thread.
---
### Author:  [@arcnmx](https://internals.rust-lang.org/u/arcnmx)
#### Post date: [February 18, 2016, 10:54pm UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/37 "2016-02-18T22:54:17Z")
> [@eddyb](#):
>
> AFAIK that was the plan, although @nikomatsakis seems to suggest that the necessary "defaulting to type parameter defaults in inference" mechanism might never be stabilized.
Wait why is that never being stabilized it's a very very very useful feature ;\_;
(though in the collections/allocator case, could libstd not instead expose it like `pub type HashMap = collections::HashMap;`? not quite ideal, though...)
> [@Amanieu](#):
>
> Another possibility for allowing code that requires memory allocation to be used in kernel/embedded environments would be to add a none system backend to libstd in addition to the existing windows and unix. This backend would simply return an error for all OS-specific functionality to indicate that it is not implemented.
This is an experiment I partially wrote the [mock bind](https://github.com/arcnmx/rust/commit/30d290f6cd456bf64a5be0930463dc7525ab4a90) backend for. You'd get compiler errors if you tried using any unimplemented features, but they'd otherwise be removed by the link/optimize step so you would get a partial libstd that worked as long as you didn't call into IO or whatever else. It can of course be implemented in pure rust instead, or have methods modified to unconditionally return `Err()`s or panic at runtime, etc.
I'd like to at some point see libstd functionality become more modular, where it may be possible to build on a platform without threads, or without networking, but still take advantage of the majority of the greater [crates.io](http://crates.io) ecosystem. These proposed changes would be a good start to making it possible and start experimenting with the techniques described above, though it certainly doesn't provide a robust final solution.
---
### Author:  [@system](https://internals.rust-lang.org/u/system)
#### Post date: [March 25, 2019, 8:25am UTC](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765/38 "2019-03-25T08:25:18Z")
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
[Previous page](https://internals.rust-lang.org/t/libsystem-or-the-great-libstd-refactor/2765.md?page=1)