Casual RFC for rust-playpen upgrades


#1

Greetings, I am currently working on making RP better. #rust-tools (hereby referred to as “We”) have identified a few must haves and few “Unofficial Goals” that would make the experience better for both the developers and the community. I know there is some drama attached to the project as it stands but I personally believe we should stick with Arch/Playpen. Playpen is good software and imvho we should continue to utilize it. Here’s the breakdown of events!

  • Phase 6P(Proper Planning Prevents Piss Poor Performance): Get the communities preferred spec for OS, server flavor, environment and any other official or unofficial goals wanted by the community.
  • Phase 1: Put RP on Arch in a Docker Container and install Deis.
  • Phase 2: Configure Deis for PoC and community pentesting.
  • Phase 3: Hand over documentation/build scripts/funny jokes to the Rust team.
  • Phase 5: Find phase 4.

###Goals(Things that need to be done to be considered feature complete):

  • Allow RP to utilize vetted external crates.
  • Allow automatic building of RP and it’s environment for many architectures.
  • Do not increase the cost of running the RP.
  • Simple/seamless upgrade/downgrades of the all components currently involved with RP.
  • Separate the IRC from the website (Will allow for load balancing and one of the unofficial goals)

###Unofficial Goals:

  • Allow trusted volunteers to distribute the load across machines/networks. (Deis allows for this)
  • Allow all external crates (When we are confident that our sandboxing methods are sufficient)

###Security Concerns:

  • Malicious crates.
  • I am relatively unknown, but am open to scrutiny and 100% transparency, no exceptions.
  • We need to utilize a hardened kernel, no exceptions.
  • Need package validation in the form of hashes and cryptography.

###Current Comments:

  • Does anybody know the GRSec crew and could get our project their stable kernel patch(Costs 6500/year)?
  • Does anybody have another preference for a hardened kernel?
  • Is anybody absolutely against Docker/Deis, if so, what alternate system would you prefer?

Note: Deis’ utilizes etcd which is “Go-lang”, anybody against this?

###Current Research:

  • Looking into the possibility of utilizing Rumprun or other immutable infrastructure on bare metal for distribution instead of Docker, the ecosystem is still new so this will take quite a bit longer. But it’s already looking like unikernels are going to be the “next big thing”.
  • Additional Goals and features of this initiative.

Thanks for your time!

-Lurkin


###Update 1:

I have decided that we will be utilizing Vagrant. We have Rust Bindings and Codegen for Cap 'n Proto. Woo! So with the limited amount of discussion we’ve had so far our current decisions are! (As of 10/22/2015 5pm Central)

  • Platform: Vagrant!
  • Containers: Sandbox.io
  • App: play-rust
  • OS:/Operating environment: Still Unknown, will update when I have made a decision. I’m going to see how small I can get the containers.

Have a great night!


Update 2: (Electric Boogaloo)

  • Platform: Vagrant + GRSec w/RBAC (We need to reach out to Brad Spengler and see if we can get stable, I don’t want to use this in production and not because I don’t think it’ll work just fine but because I feel scummy contributing to the reason it was made unavailable to us in the first place. Since this is a PoC I will use this, maybe somebody in the community is already a sponsor, maybe the higher ups can get a dialog going I lost sleep thinking about this last night. I just know I’m not in a position to say we should use this in production. We should discuss delegating who should reach out to @GRSecurity again all imvho, if we do it right the first time it’ll be better for us in the end. Edit @ 9:02am 10/25 2015)
  • Container: *
  • App: Play-Rust

Vagrant will allow us to scale our infrastructure needs, GRSec is widely trusted and RBAC is a great tool. The containers on the other hand we have a bit of an option and I will try to lay out the pro’s and con’s of each option.

####Unikernel(runrump) A newer technology that allows you to take your posix application and run it on bare metal(KVM, Xen, AWS) It is POSIX compliant. When you “bake” your runrump kernel it only keeps the system calls it needs, I’m in the middle of trying this out now because It’s awesome and will allow us to run the rust-playground anywhere. This opens up so many options it’s hard to explain them all, if it works I’ll include a .iso for testing :smile:

####Sandstorm(Container Technology) Proven technology with more overhead than a unikernel. Sandstorm has RustC bindings for Cap 'n Proto and even Code-gen capabilities. What more could we possibly ask for? One of my personal goals is to make it more cost effective to run the playground than it is right now, that is the only reason we are not going with this from the get-go. If it’s not doable to package rust-playground in a unikernel this will be the option we go for (and we’re not settling by any means)

If we do use sandstorm we will make sure to continue to utilize application level sandboxing, I don’t think we’ll need application level sandboxing if we utilize the rumpkernel but we will see.

###New UnOfficial Goals: @alexcrichton Local rust-playground that preferably does not utilize cygwin on windows and does not require a full VM to utilize. (If rumprun works this becomes official)

Enjoy your night!

######Things we are no longer considering:

  • Deis
  • Docker

###Update 3: Goooooood Morning Rustaceans!

I have identified the first changes we will be making to the playground, woo! First we will be creating a rust binary that can replace -cd in packstrap.sh! Pretty great? Great! Next we will be creating a custom version of rustup.sh that will get rustc from a private local repository that ALSO includes the vetted crates we were referring to earlier. This will mean (still assuming rumprun will work for us, but the plan holds for sandstorm as well) that we can essentially bake images of playground with any feature anybody wants to add for any reason! In the future it will be possible to bake each instance with all the crates that were available the day before! Not that every crate is automatically included, but that they would all be available for “baking”.

I wanted to brainstorm a bit about what @alexcrichton mentioned regarding the importance of having RP as a local tool. After a good nights sleep here’s my take on what should be possible locally.

  • Useful for interactive presentations
  • Useful for classroom interactions/ structured tutorials
  • The audience ought to be able to follow along with the presenter without setup.

The last point I want to elaborate on more. I think it would be pretty nifty to be able to host your own playground from your presentation machine. You can bake the image before hand with the required crates (say you need nix crate) and then have an environment already available to work in. Those that want to tinker further can get copies of the playground in the state of the presentation from the web, those that are there to learn a bit or politely follow along can do so without any major setup. This would require a server on the presentation machine to host the playground web interface for the local network and that’s about it. The poor machine would take a beating compiling for the entire room but in a situation where subject matter comes first and time a very close second it seems this kind of “local reproducibility” is maybe something worth looking into. Or maybe I’ve had too much coffee. Thoughts?

Quick note to myself: The server has 2 functions, to distribute copies of playpen to those who have the hardware/software to run it, and as a web server for playground itself for folks coming with mobile/lighter hardware.

######Also on the radar at some point is replacing irc.py with irc.rs from here. But don’t tell anybody because it’s a large undertaking for a non-programmer/programmer in training.

Finally we will be adding -n and -b switches to the IRC playground for nightly and beta respectively, if there is a push for nightly: and beta: Then I will include both but I think “-” is more effective.


###Update 5: Good morning internals :slight_smile: I’ve spent the weekend rewriting playbot in rust. I’m not ready to show code (I am making progress though). Here’s an executive view rundown of what’s happening and how.

We’re creating a bin file with playbot(Brought to us by aatxe’s irc crate), multirust-rs. we will then be wrapping this bin with nginx into the Rump Kernel which will allow execution via KVM, Xen, and VirtualBox with little issue. Multirust is the real key piece of infrastructure here, we will be using it to specify toolchains for both the playground and playbot. The only downside is that to add more toolchains to multirust’s option you must recompile the rump kernel, this is more of a feature imo(playing into rumpkernels ideals of immutable infrastructure).

Every previously goal mentioned is on the table now, it should even play well with your existing buildbot infrastructure. In the end you will have a heavily documented binary that prepares the environment for both the Rump Kernel and Playground to allow safe execution of arbitrary code.


#2

I’ve been playing with Rumprun, but yeah, it’s still quite new.


#3

Thank you for taking the initiative on this much-needed project! Just a couple thoughts:

  • I’d like more detail on what http://docs.deis.io/en/latest/managing_deis/security_considerations/ means by “Deis is not suitable for multi-tenant environments or hosting untrusted code.”. It might mean “containers alone aren’t sandboxes”, which is sane and reasonable, but could potentially refer to some other kind of security weakness.

  • sandstorm.io is also tackling challenge of hardening container-like things, and some of their techniques are listed at https://docs.sandstorm.io/en/latest/using/how-it-works/

  • I would never rule out the right tool for the job based on the language it’s written in. If there was a competitive Rust alternative, I’d prefer it, and it’ll be interesting to see what containerization layer Redos develops as it evolves into a production-ready system. For now, though, a conveniently reproducible (and maybe even testable? :smile: ) form of the playground would enable users to improve the play.r-l.o experience in a positive feedback loop that we don’t have at the moment.


#4

I’d like more detail on what http://docs.deis.io/en/latest/managing_deis/security_considerations/ means by “Deis is not suitable for multi-tenant environments or hosting untrusted code.”. It might mean “containers alone aren’t sandboxes”, which is sane and reasonable, but could potentially refer to some other kind of security weakness.

Thanks for the feedback! This Github issue, seems to go into more detail about the statement they make in their security considerations. They are saying that Deis does not yet offer technologies that ensure containers cannot talk to one another. They go on to say that that warning is not implying that everything will break if you do run a multi tenant environment, only that Deis itself does not mitigate any risk associated with multi tenant infrastructure (They are planning on adding it for 2.0) To mitigate this risk we will be utilizing VPN to provide network segregation between the control and data planes. We will be utilizing iptables rules to ensure this segregation is maintained on an OS, Container, and Host level. We will utilize VPN to connect our control plane to our other potential trusted volunteers with idle hardware. This is pretty dangerous, we need to seriously trust our volunteers and their practices. The warning is at the very top of the first page because this is non trivial work meaning that it’s not a weekend project to implement segregation.

https://docs.sandstorm.io/en/latest/using/how-it-works/

Decided on this instead of Deis, Vagrant is better for our situation after considering the information in update 1.

###Current Security Concerns:

VPN security, especially if we’re planning on reaching our distributed infrastructure goal. While the control plane is not the keys to the kingdom a ton of damage can be done to the network if the control plane is compromised and undetected.

Me, still unvetted, I still wouldn’t vouch for myself if I was y’all but I will do everything in my power so that I do not have to be trusted. Please let me know how I can ensure transparency and accountability beyond excessive comments/extreme documentation. I would even be willing to screencast my terminals for every documented process. In this way if somebody repeats the documentation they should be able to reproduce an exact hash of the build ensuring I haven’t hidden any sort of nastiness away from view. Anything I can do to future proof this project is my pleasure to accomplish.

Cryptographically signed containers are needed, it’s been beaten to death that rust is a bit lacking in the dept of signing their stuff with PGP but we should be setting and exceeding the standards all this automated compiling and building is going to cause us (can I say us?) problems in the future. Edit: s/is a bit lacking/is missing the appropriate documentation regarding where to find said wonderful things. :smile:

I would never rule out the right tool for the job based on the language it’s written in. If there was a competitive Rust alternative, I’d prefer it, and it’ll be interesting to see what containerization layer Redos develops as it evolves into a production-ready system. For now, though, a conveniently reproducible (and maybe even testable? :smile: ) form of the playground would enable users to improve the play.r-l.o experience in a positive feedback loop that we don’t have at the moment.

Not sure what you mean by testing but your wish is my command…as soon as you elaborate :smile: I should have more answers for the containers shortly.


#5

https://pgp.mit.edu/pks/lookup?op=vindex&search=0x85AB96E6FA1BE5FE

and the .asc files in http://static.rust-lang.org/dist/


#6

I’m excited to see some movement here, so thanks for taking the initiative here @Lurkin!

My only main concerns with today’s setup is the reproducibility locally for development and for others to take advantage of (e.g. it’s really hard to change anything). If we’ve got that solved them I’m pretty amenable to other changes.

I have not personally heard of Deis before, but in terms of deployment EC2 will probably work for us in prod although I’m not sure how connected those two parts are…