A beginner’s guide to writing your DevOps tools in Rust.
In this blog post we’ll cover some basic DevOps use cases for Rust and why you would want to use it. As part of this, we’ll also cover a few common libraries you will likely use in a Rust-based DevOps tool for AWS.
If you’re already familiar with writing DevOps tools in other languages, this post will explain why you should try Rust.
We’ll cover why Rust is a particularly good choice of language to write your DevOps tooling and critical cloud infrastructure software in. And we’ll also walk through a small demo DevOps tool written in Rust. This project will be geared towards helping someone new to the language ecosystem get familiar with the Rust project structure.
If you’re brand new to Rust, and are interested in learning the language, you may want to start off with our Rust Crash Course eBook.
Rust is a systems programming language focused on three goals: safety, speed, and concurrency. It maintains these goals without having a garbage collector, making it a useful language for a number of use cases other languages aren’t good at: embedding in other languages, programs with specific space and time requirements, and writing low-level code, like device drivers and operating systems.
The Rust Book (first edition)
Rust was initially created by Mozilla and has since gained widespread adoption and support. As the quote from the Rust book alludes to, it was designed to fill the same space that C++ or C would (in that it doesn’t have a garbage collector or a runtime). But Rust also incorporates zero-cost abstractions and many concepts that you would expect in a higher level language (like Go or Haskell). For that, and many other reasons, Rust’s uses have expanded well beyond that original space as low level safe systems language.
Rust’s ownership system is extremely useful in efforts to write correct and resource efficient code. Ownership is one of the killer features of the Rust language and helps programmers catch classes of resource errors at compile time that other languages miss or ignore.
Rust is an extremely performant and efficient language, comparable to the speeds you see with idiomatic everyday C or C++. And since there isn’t a garbage collector in Rust, it’s a lot easier to get predictable deterministic performance.
What makes Rust unique also makes it very useful for areas stemming from robots to rocketry, but are those qualities relevant for DevOps? Do we care if we have efficient executables or fine grained control over resources, or is Rust a bit overkill for what we typically need in DevOps?
Yes and no
Rust is clearly useful for situations where performance is crucial and actions need to occur in a deterministic and consistent way. That obviously translates to low-level places where previously C and C++ were the only game in town. In those situations, before Rust, people simply had to accept the inherent risk and additional development costs of working on a large code base in those languages. Rust now allows us to operate in those areas but without the risk that C and C++ can add.
But with DevOps and infrastructure programming we aren’t constrained by those requirements. For DevOps we’ve been able to choose from languages like Go, Python, or Haskell because we’re not strictly limited by the use case to languages without garbage collectors. Since we can reach for other languages you might argue that using Rust is a bit overkill, but let’s go over a few points to counter this.
To elaborate on some of these points a bit further:
For DevOps it’s also worth mentioning the (relative) ease with which you can port your Rust code across different architectures and different OS’s.
Using the official Rust toolchain installer rustup, it’s easy to get the
standard library for your target platform.
Rust supports a great number of platforms
with different tiers of support.
The docs for the rustup tool has a section
covering how you can access pre-compiled artifacts for various architectures.
To install the target platform for an architecture (other than the host platform which is installed by default)
you simply need to run rustup target add:
$ rustup target add x86_64-pc-windows-msvc
info: downloading component 'rust-std' for 'x86_64-pc-windows-msvc'
info: installing component 'rust-std' for 'x86_64-pc-windows-msvc'Cross compilation is already built into the Rust compiler by default.
Once the x86_64-pc-windows-msvc target is installed you can build for Windows
with the cargo build tool using the --target flag:
cargo build --target=x86_64-pc-windows-msvc(the default target is always the host architecture)
If one of your dependencies links to a native (i.e. non-Rust) library, you will
need to make sure that those cross compile as well. Doing rustup target add
only installs the Rust standard library for that target. However for the other
tools that are often needed when cross-compiling, there is the handy
github.com/rust-embedded/cross tool.
This is essentially a wrapper around cargo which does all cross compilation in
docker images that have all the necessary bits (linkers) and pieces installed.
A key unique feature of Rust is that it doesn’t need a runtime or a garbage collector. Compare this to languages like Python or Haskell: with Rust the lack of any runtime dependencies (Python), or system libraries (as with Haskell) is a huge advantage for portability.
For practical purposes, as far as DevOps is concerned, this portability means that Rust executables are much easier to deploy than scripts. With Rust, compared to Python or Bash, we don’t need to set up the environment for our code ahead of time. This frees us up from having to worry if the runtime dependencies for the language are set up.
In addition to that, with Rust you’re able to produce 100% static executables for
Linux using the MUSL libc (and by default Rust will statically link all Rust code).
This means that you can deploy your Rust DevOps tool’s binaries across your Linux
servers without having to worry if the correct libc or other libraries were
installed beforehand.
Creating static executables for Rust is simple. As we discussed before, when discussing
different OS targets, it’s easy with Rust to switch the target you’re building against.
To compile static executables for the Linux MUSL target all you need to do is add
the musl target with:
$ rustup target add x86_64-unknown-linux-muslThen you can using this new target to build your Rust project as a fully static executable with:
$ cargo build --target x86_64-unknown-linux-muslAs a result of not having a runtime or a garbage collector, Rust executables can be extremely small. For example, there is a common DevOps tool called CredStash that was originally written in Python but has since been ported to Go (GCredStash) and now Rust (RuCredStash).
Comparing the executable sizes of the Rust versus Go implementations of CredStash, the Rust executable is nearly a quarter of the size of the Go variant.
| Implementation | Executable Size |
|---|---|
| Rust CredStash: (RuCredStash Linux amd64) | 3.3 MB |
| Go CredStash: (GCredStash Linux amd64 v0.3.5) | 11.7 MB |
Project links:
This is by no means a perfect comparison, and 8 MB may not seem like a lot, but consider the advantage automatically of having executables that are a quarter of the size you would typically expect.
This cuts down on the size your Docker images, AWS AMI’s, or Azure VM images need to be – and that helps speed up the time it takes to spin up new deployments.
With a tool of this size, having an executable that is 75% smaller than it would be otherwise is not immediately apparent. On this scale the difference, 8 MB, is still quite cheap. But with larger tools (or collections of tools and Rust based software) the benefits add up and the difference begins to be a practical and worthwhile consideration.
The Rust implementation was also not strictly written with the resulting size of the executable in mind. So if executable size was even more important of a factor other changes could be made – but that’s beyond the scope of this post.
Rust is very fast even for common idiomatic everyday Rust code. And not only that it’s arguably easier to work with than with C and C++ and catch errors in your code.
For the Fortunes benchmark (which exercises the ORM, database connectivity, dynamic-size collections, sorting, server-side templates, XSS countermeasures, and character encoding) Rust is second and third, only lagging behind the first place C++ based framework by 4 percent.
In the benchmark for database access for a single query Rust is first and second:
And in a composite of all the benchmarks Rust based frameworks are second and third place.
Of course language and framework benchmarks are not real life, however this is still a fair comparison of the languages as they relate to others (within the context and the focus of the benchmark).
Source: https://www.techempower.com/benchmarks
For medium to large projects, it’s important to have a type system and compile time checks like those in Rust versus what you would find in something like Python or Bash. The latter languages let you get away with things far more readily. This makes development much “faster” in one sense.
Certain situations, especially those with involving small project codebases, would benefit more from using an interpreted language. In these cases, being able to quickly change pieces of the code without needing to re-compile and re-deploy the project outweighs the benefits (in terms of safety, execution speed, and portability) that languages like Rust bring.
Working with and iterating on a Rust codebase in those circumstances, with frequent but small codebases changes, would be needlessly time-consuming If you have a small codebase with few or no runtime dependencies, then it wouldn’t be worth it to use Rust.
We’ll briefly cover some of the libraries typically used for an AWS focused DevOps tool in a walk-through of a small demo Rust project here. This aims to provide a small example that uses some of the libraries you’ll likely want if you’re writing a CLI based DevOps tool in Rust. Specifically for this example we’ll show a tool that does some basic operations against AWS S3 (creating new buckets, adding files to buckets, listing the contents of buckets).
For AWS integration we’re going to utilize the Rusoto library. Specifically for our modest demo Rust DevOps tools we’re going to pull in the rusoto_core and the rusoto_s3 crates (in Rust a crate is akin to a library or package).
We’re also going to use the structopt crate for our CLI options. This is a handy, batteries included CLI library that makes it easy to create a CLI interface around a Rust struct.
The tool operates by matching the CLI option and arguments the user passes in
with a match expression.
We can then use this to match on that part of the CLI option struct we’ve defined and call the appropriate functions for that option.
match opt {
Opt::Create { bucket: bucket_name } => {
println!("Attempting to create a bucket called: {}", bucket_name);
let demo = S3Demo::new(bucket_name);
create_demo_bucket(&demo);
},This matches on the Create
variant of the Opt enum.
We then use S3Demo::new(bucket_name) to create a new S3Client which we can
use in the standalone create_demo_bucket function that we’ve defined
which will create a new S3 bucket.
The tool is fairly simple with most of the code located in src/main.rs
Before you build the code in this project, you will need to install Rust. Please follow the official install instructions here.
The default build tool for Rust is called Cargo. It’s worth getting familiar with the docs for Cargo but here’s a quick overview for building the project.
To build the project run the following from the root of the git repo:
cargo buildYou can then use cargo run to run the code or execute the code directly
with ./target/debug/rust-aws-devops:
$ ./target/debug/rust-aws-devops
Running tool
RustAWSDevops 0.1.0
Mike McGirr <[email protected]>
USAGE:
rust-aws-devops <SUBCOMMAND>
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
SUBCOMMANDS:
add-object Add the specified file to the bucket
create Create a new bucket with the given name
delete Try to delete the bucket with the given name
delete-object Remove the specified object from the bucket
help Prints this message or the help of the given subcommand(s)
list Try to find the bucket with the given name and list its objects``Which will output the nice CLI help output automatically created for us
by structopt.
If you’re ready to build a release version (with optimizations turn on which will make compilation take slightly longer) run the following:
cargo build --releaseAs this small demo showed, it’s not difficult to get started using Rust to write DevOps tools. And even then we didn’t need to make a trade-off between ease of development and performant fast code.
Hopefully the next time you’re writing a new piece of DevOps software, anything from a simple CLI tool for a specific DevOps operation or you’re writing the next Kubernetes, you’ll consider reaching for Rust. And if you have further questions about Rust, or need help implementing your Rust project, please feel free to reach out to FP Complete for Rust engineering and training!
Want to learn more Rust? Check out our Rust Crash Course eBook. And for more information, check out our Rust homepage.
Subscribe to our blog via email
Email subscriptions come from our Atom feed and are handled by Blogtrottr. You will only receive notifications of blog posts, and can unsubscribe any time.