Extending existing functionality


#1

I think Rust might currently be missing a convenient way to allow the addition of functionality to existing code, especially library code. As a library writer you currently have two options: make something (attribute/function) public, which allows use by others and makes it part of the API, or make it private, which completely hides it away, inaccessible, for everyone. There is no option to say something like “this is not the public API, but you can use it if you know what you’re doing.” C++ has this in the form of protected members, Ruby has this in the form of extending classes (either directly or through modules).

The current status quo generally leaves several options: 1) try to use the public API, which might be impossible at worst or sometimes horribly slow at best 2) modify the library, which is not very portable, especially in the case of the standard library 3) reimplement the original code, which seems silly if the original code is good.

How could this be properly supported in Rust?

In case you’re still wondering what I’m talking about, here’s an example of something someone might want to do that is currently not possible:

use std::intrinsics::cttz32;
use std::collections::{Bitv,BitvSet};

pub trait LowestBit
{
    fn lowest_bit(&self) -> Option<usize>;
}

impl LowestBit for Bitv
{
    fn lowest_bit(&self) -> Option<usize>
    {
        // error: field `storage` of struct `collections::bit::Bitv` is private
        for (idx,&word) in self.storage.iter().enumerate()
        {
            if (word==0) { continue }
            return Some(idx*32+(unsafe {cttz32(word) as usize}));
        }
        return None;
    }
}

fn main() {
    let mut v=BitvSet::new();
    v.insert(1242);
    println!("{}",v.get_ref().lowest_bit().unwrap());
}

Yes, one can implement lowest_bit a different way, but at the expense of performance.

(You might have seen this on GitHub issue tracker - I realize the bug tracker is not an appropriate place to discuss this but this forum is actually not linked from the rust-lang.org homepage)


#2

You could mark this kind of public but at best not used functions as unstable (or experimental in a unstable API). This would perfectly convince the meaning for many cases. Nevertheless it doesn’t fit the public and stable but only meant for APIs extending this API case.

Maybe adding some kind of internal or api_extension annotations would make sense? This could generate a warnings as long as the using function/module/impl is not marked with #[allow(internal)] or #[extending_api].

I think having such a mechanism witch is decoupled from stability (marked functions can still be unstable etc.) would be nice in some cases.

(BTW. I would prefer the extending_api version)


#3

unsafe is exactly what you are looking for.


#4

unsafe doesn’t let you access private members, or are you suggesting that it should?


#5

I disagree. unsafe is about type safety. OP wants a mechanism for integrating with unstable code. They’re two separate concepts, and I’d rather have two separate ways of representing this in the language.


#6

You can use unsafe to cast to a struct with the same fields, and adjust the memory.


#7

Ah, cool idea. Now we just need a macro or such that makes an exact copy of a struct definition, would that be possible?

I’m having problems envisioning this; in particular I don’t think you can do a type lookup at the appropriate time; see e.g. https://stackoverflow.com/questions/27185147/rust-syntax-extension-and-traits


#8

You can currently, but this may break any time (e.g. with struct layout randomization).


#9

At compile time, you should be able to do this deterministically, yes?

Actually, I’m having trouble with this: the same issue that is preventing access in the example case above is preventing compile-time checks on the offset and size of such members.


[Pre-RFC] inherent methods from any crate
#11

In a similar vein, I suggested an annotation to explicitly ignore field visibility a little while ago. I like this approach (or requiring unsafe) because it’s less boilerplate to write/maintain than casting to a “public version” of the same struct.


#12

I’m looking at modifying the privacy checker itself. It is almost trivial to base the access check on attributes of the current item. The following diff will make the code in the first post work if you add #[private_access] before the impl.

diff --git a/src/librustc/middle/privacy.rs b/src/librustc/middle/privacy.rs
index aa37c2f..0f30a69 100644
--- a/src/librustc/middle/privacy.rs
+++ b/src/librustc/middle/privacy.rs
@@ -693,6 +693,7 @@ impl<'a, 'tcx> PrivacyVisitor<'a, 'tcx> {
                    span: Span,
                    id: ast::DefId,
                    name: FieldName) {
+        if ty::has_attr(self.tcx,ast::DefId{krate:ast::LOCAL_CRATE,node:self.curitem},"private_access") { return }
         let fields = ty::lookup_struct_fields(self.tcx, id);
         let field = match name {
             NamedField(ident) => {

Of course, this does not yet have support for methods and paths and I would like to have more granularity in selecting what would be permissible. Unfortunately, you can’t but fully-qualified names in an attribute: #[private_access(Bitv)] is ok but #[private_access(std::collections::Bitv)] is not. One idea I have is to allow such an attribute only on an impl TraitName for StructName and limit the access only to private fields/members of StructName. How would people like this to work?


#13

For a case I particularly ran into, extending std::io::Command with a method that can modify the command to wrap the call in ‘sudo’, just having access to the private fields would be sufficient. But, it seems more generally useful to gain access to the private/internal methods of the type.


#14

I’m soliciting feedback on this draft RFC https://github.com/jethrogb/rfcs/blob/topic/visibility-override/text/0000-visibility-override.md