Option to add replacement std dependency to subdependency's for no_std platforms

tools and infrastructure
1

I figured I'd try here first before raising a github feature request for cargo.

My understanding is typically with embedded platforms there's no standard library. Part of the reason for this is that std includes operating system functionality which isn't needed. But also std covers use of types and functionality assicoated with the heap such as String which isn't available by default.

This hasn't stopped people from trying though to gain access to some of the code libraries already out there. I've seen a few workarounds to this

There's also been some use of cargo +nightly -Zbuild-std=std build
but that's considered something of a hack and not supported / requires nightly.
I think it's also been broken for a few months now as well due to thread_local

The end result seems to be lots of different kinds of hacks and fudges as a way of working around the problem if you want to use a library that references std in some small way. Some people you say you shouldn't be using a heap on an embedded device in the first place. Other people like myself like the idea of being able to use the device however we want.

For a top level crate that your working on, you can do something like this for example using the nostd library

[dependencies]
std = { package = "nostd", version = "0.1" }

We still have to add #![no_std] to the top of main.rs and we have to manually add in any use statements as this fake std won't be imported / use aliased automatically

use std::string::String;
use std::vec::Vec;

But it does seem the cleanest option so far assuming we use a heap (the rp2350 has 512Kb of memory which can be extended up to 16Mb with PSRAM)

Where this tends to fall down however is a dependency / library that's also trying to use String or some other heap related type. Currently there's no way to feed in a substitute std library into that dependency without forking it and making changes to it's Cargo.toml file.
I looked into build.rs but that doesn't have the functionality to perform any form of substitution Programatic access to cargo as a library might be one option but that's not guaranteed to have a stable API I think.

As a suggestion, perhaps a simple option that allows adding a dependency to all packages underneath the current crate (subdepends) via something like a special dependencies.global option

[dependencies.global]
std = { package = "nostd", version = "0.1" }

Or even something that's more specific just for std replacement.

This wouldn't be the only thing you'd need to do though, typically with the normal std library, things tend to be auto imported (you can use String without use std::string::String; for example) So a code change sonewhere else (not sure if it would be under rustc) would be also needed to try and handle this somehow under #![no_std] environments.

The idea is to gain greater compatibility with existing libraries that were never written with no_std in mind when they were originally written. Also to find a more consistent way to swap out std functionality across the board (for all subdepends) via something like a custom std crate that sits next to the main project.

2

Have you taken a look at std-aware cargo?

3

I'm no_std dev myself, but unfortunately I couldn't understand what is your proposal. Like, if some no_std (and no alloc) library is written like no_std library, what is going to change for you as a user? Don't you anyway need to change the library to actually make use of std?

And about alloc, there is a good reason to avoid it - on devices without MMU, your heap will soon become fragmented and your program will crash (OOM). This is not an issue for non-realtime 64 bit systems with MMU because fragmentation is limited to a single page, anything higher can be remapped freely.

4

To try and explain, lets say I'm writing a project that uses #![no_std]
I depend on a library written by someone else that uses std::string::String;
This library is not aware of anything to do with no_std or embedded platforms so fails to build because std::string::String; is not visible.
In order to avoid making changes to this library (because there might be lots of them)

  1. I specify to cargo in my own project that any dependencies I'm adding in should have they're dependencies (subdepends) modified to include a custom std library implementation, something like the following
[dependencies.global]
std = { package = "nostd", version = "0.1" }

Or Maybe

[dependencies.global]
std = { package = "my_custom_std_lib", version = "0.1", path = "../my_custom_std_lib" }
  1. We have a feature or magic somewhere in the compiler level that allows us to automatically add use statements to the top of every source, including all the libraries we've added in as depends
use std::string::String;

In a sense emulating or replacing the std library (or just parts of it) with our own custom implementation. How the compiler magic would work I'm not sure maybe something like replacing #![no_std] with #![custom_std]

#![custom_std]
mod special_global {
    use std::string::String;
}

Thanks for the advise on alloc btw

5

If a crate is using std::string::String then replacing std with nostd won't give you any benefit, since both crates' String are just re-export of alloc::string::String. The only non-reexported part of nostd is the io module.

6

Sorry not sure I explained things very well. What I'm aiming for is a way to trick dependencies into using a different but similar implementation of std in a limited fashion (since the normal std doesn't exist) without the need to change any of they're code

So to give another example Lets say we have a top level crate called "myproject" and it has a cargo dependency called "mydepend"
Both of these are for an embedded target setup with #![no_std]

Use Magic shenanigans

Within the top level "myproject" we have some magic that looks like this

#![custom_std]
mod special_global {
    use nostd::string::String;
}

Now within "mydepend" the following code is transformed from this

fn example() {
    println!("Hello, world!");
    let _hello = String::from("Hello, world!");
}

To this

use nostd::string::String;

fn example() {
    println!("Hello, world!");
    let _hello = String::from("Hello, world!");
}

This is the first step

Create Dependency shenanigans

Now that dependency has a line of code at the top to use nostd::string::String; in place of the original String definition which is missing because of #![no_std]

But it has no dependency on that library so it's going to fail to build.
So next we need some magic within "myproject" Cargo.toml file to pass down some information to the "mydepend" crate that it needs some additional dependency

[package]
name = "myproject"
edition = "2024"
version = "0.1.0"

[dependencies.mydepend]
version = "1.0.0"
# Not sure if this is the best way to do it but you get the idea
add_depends = [
  nostd = "0.1"
]

# Or maybe something special like this for all dependencies
[dependencies.global]
add_depends = [
  nostd = "0.1"
]

Not sure if this makes sense but the idea is to allow for the use of existing libraries by effectively building up parts of your own std implementation (it doesn't need to be nostd, it could be something totally custom locally) for an embedded platform.

7

I've understood that, but this is just how you're trying to solve a problem, it is not (or, should not be) the goal itself.

What issue are you actually trying to solve?

8

I don't think the Linux kernel is using cargo, but it replaces std AFAIK and has also looked to import external crates (not sure if any are actually merged or not). AFAIK, it does have different allocation API patterns, but it might still have useful tidbits for something like this.

9

I'm trying to import a library that might not be no_std aware as a dependency then add in whatever functionality it's dependent on from std to get it to work without having to make any changes to that library

Edit I stumbled across this which also discusses different ways of patching the sysroot

1 Like
10

So you want a way to use an existing library, where that library is not already no_std and uses std APIs.

Your proposed solution is a way to patch that library to resolve std APIs to some crate that isn't the std provided by Rust.

I think a much better option would be to just patch that library to make it no_std in the first place.

4 Likes
11

I think the problem with that approach is it defeats the purpose of abstraction.
When you use an type such as a String or Vec, ideally you know it works but the implementation is hidden. All you know is that it has a set of methods and an expected behaviour. It's part of the reason for an interface, in a sense your plugging in an alternative implementation.

Typically you may have lots and lots of libraries in the wider world. Patching each one individually for a nonstd use case seems counter productive long term. Although I admit there will be use cases where that is needed. I think this is part of the reason for all the different hacks out there

I did run across GitHub - japaric/xargo: The sysroot manager that lets you build and customize `std` recently but it's in maintenance mode probably due to the -Zbuild-std feature (which currently has issues in my case)

I'm going to do some more digging though anyway since there's a lot I admit I don't understand
yet (such as heap fragmentation). Typically when you get to a certain level of complexity it's sometimes better to do something via an api rather than configuration settings.

I'm now wondering if a way to do this would be to tap into cargo as a library via a form of wrapper on top although from what I can gather I don't think cargo has a stable or standardised api
but it might offer a way to introduce an example or suggested way of doing something before introduced proper via an rfc or whichever means is needed

12

I think that we should be looking for ways to make it easier for libraries to be no_std compatible by default — make it not "nonstandard" any more.

3 Likes
13

A clippy lint may be a good place to start:

14

Since this is the case, it seems that replacing std mostly solves the case of automatically patching crates written using std:: to instead use alloc:: and core::. It doesn't really assist in making any code no_std compatible that isn't in spirit already compatible, it only makes the burden of making it compatible in body easier. In effect, it collapses the facade of std similar to the eventual goal of std as a crate making the std/alloc/core split into feature flags of std just like for any other crate.

1 Like
15

This is not true. For example: std::f32 has sin and cos, core::f32 does not. I don't know if that is the only exception apart from io in std, but it is one I know of.

16

The nostd crate doesn't export sin or cos.