Pre-RFC: Extend item visibility when used by macro

language design
1

Summary

Allow item to have extended visibility, when used by selective macros, including "Macros by Example" and "Procedural Macros" to ensure private APIs used in public macros are truly private.

Motivation

How we do this today

Currently, crate authors uses some hacks to make macros in public API to access "private" APIs.

This is done by prefix them by "pub" keyword and "_" at the beginning of name, and #[doc(hidden)] attribute.

Using anyhow crate as an example (With some simplification):

#[macro_export]
macro_rules! anyhow {
    ($msg:literal $(,)?) => {
        $crate::__private::must_use({
            let error = $crate::__private::format_err($crate::__private::format_args!($msg));
            error
        })
    };
}

// Not public API. Referenced by macro-generated code.
#[doc(hidden)]
pub mod __private {
    pub use core::format_args;
    #[doc(hidden)]
    #[inline]
    #[cold]
    #[must_use]
    pub fn must_use(error: Error) -> Error {
        error
    }
}

The macro anyhow! calls APIs in the __private, which is not in the public API, but it has public visibility, means the user can technically uses APIs in __private module, without using anyhow! macro.

Guide-level explanation

Extension to the visibility syntax

Allow word "macro" to be used in visibility syntax

pub(crate, macro global in crate)

This item has crate visibility by default, but has global visibility, when used inside any "Macros by Example" defined in the current crate, or code generated by any "Procedure Macros" re-exported in the current crate

pub(self, macro global in super)

This item has self visibility (Only visible in the current module) by default, but has global visibility, when used inside any "Macros by Example" defined in the parent module, or code generated by any "Procedure Macros" re-exported in the parent module

pub(self, macro global in crate::macros)

This item has self visibility (Only visible in the current module) by default, but has global visibility, when used inside any "Macros by Example" defined in the crate::macros module, or code generated by any "Procedure Macros" re-exported in the crates::macros module

pub(macro global in crate::macros)

Not allowed. Default visibility must be specified

pub(global)

Allowed. Equivalent to pub

pub(global, macro self in crate::macros)

Not allowed. Macro visibility must be as public as the default visibility.

Procedure Macros

Visibility is only extended when at least one of procedure macro import chain, is re-exported inside namespace of the visibility declaration.

// Crate foo

pub(self, macro global in crate)
struct Foo {

}

pub use foo_derive::CreateFoo;

// Crate foo_derive
extern crate proc_macro;
use proc_macro::TokenStream;
#[proc_macro_derive(CreateFoo)]
pub fn create_foo(_item: TokenStream) -> TokenStream {
    "fn create_foo() -> ::foo::Foo {
      ::foo::Foo{}
    }".parse().unwrap()
}

The following works

// Other crate that uses `foo` or `foo_derive` crate

#[derive(foo::CreateFoo)]
struct Bar;

The following does not compile, because anything in the import chain of foo_deriv::CreateFoo is not imported in the foo crate. Note that the visibility of struct Foo is declared as pub(self, macro global in crate),

// #[derive(foo_derive::CreateFoo)]
// struct Bar;

// ERROR: `foo::Foo` is private

Reference-level explanation

TODO

Drawbacks

  1. The syntax is complex, and may be hard to understand

More TODO

Rationale and alternatives

TODO

Prior art

Rust standard library uses #[allow_internal_unstable] to access private items

#[cfg(all(not(no_global_oom_handling), not(test)))]
#[macro_export]
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_diagnostic_item = "vec_macro"]
#[allow_internal_unstable(rustc_attrs, liballoc_internals)]
macro_rules! vec {
    () => (
        $crate::__rust_force_expr!($crate::vec::Vec::new())
    );
    ($elem:expr; $n:expr) => (
        $crate::__rust_force_expr!($crate::vec::from_elem($elem, $n))
    );
    ($($x:expr),+ $(,)?) => (
        $crate::__rust_force_expr!(<[_]>::into_vec(
            // This rustc_box is not required, but it produces a dramatic improvement in compile
            // time when constructing arrays with many elements.
            #[rustc_box]
            $crate::boxed::Box::new([$($x),+])
        ))
    );
}

$crate::vec::from_elem is a private item, but it can be used by other crates by the vec! macro, due to #[allow_internal_unstable(rustc_attrs, liballoc_internals)]

Unresolved questions

  1. How should it interact with Procedural Macros? TODO

Future possibilities

TODO

2

The "macros 2.0" syntax using the macro keyword solves this with "privacy hygiene," in that privacy is checked for macro-expanded code in the module the macro is defined in (or more accurately, for each token, in the module that token was written in) instead of the context where the macro is expanded.

The pin! macro actually originally used this, but switched to use allow_internal_unstable because that's a more well understood system than "hygiene 2.0" which is still more adhoc than designed at the moment.

6 Likes
3

I will further note that there is an ongoing effort to form a macros WG to revive work on this, with hygiene/name resolution explicitly included as part of it. An MCP is currently open.

4 Likes
4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.