Getting the type signature right was 99% of the work in implementing
HLists in Frunk.
Here’s what I’ve learnt along the way: what works, and what doesn’t work (and why).
As you may already know, Rust eschews the now-mainstream object-oriented model of programming (e.g. in Java, where behaviour for a type is added to the type/interface definition directly) in favour of a typeclass-like approach (e.g. in Haskell where you can ad-hoc add behaviour to a type separate from the type definition itself). Both approaches have their merits, and indeed, some languages, such as Scala, allow for a mix of both.
For those coming from the OOP school of programming, Rust’s system of adding behaviour to types might be daunting to come to grips with. At a glance, it might not be obvious how to get things done, especially when what you want to build goes beyond implementing
Eq. If your abstraction has a certain degree of type-level recursiveness, it might be even harder to see the light at the end of the tunnel, and the lack of online material covering that sort of thing doesn’t help.
As a Scala guy with Haskell knowledge, I’m no stranger to typeclasses, but it took me a while and several failed attempts to figure out how to implement the following:
- Plucking out a value by type from an HList and getting back the remainder **
- Sculpting an HList into another shape, and getting back the remainder (in the case where we only want a smaller subset than the original) **
Of course, the type signature of the finished product can be intimidating !
In this post, I’ll briefly introduce Rust’s trait system and present my mental model for writing trait implementations that deal with type-level recursion. To do so, I will go through how
sculpt() were written in Frunk, as well as recount some of my failed approaches so you can learn from my mistakes.
Hopefully, by the end of it, you’ll be able to look at signatures like the one above and not go “WTF”, but rather, “FTW”.
LabelledGeneric? How does one encode type-level Strings in Rust? What is a labelled HList?
Hold on, let’s take a step back.
In a previous post about implementing
Generic in Rust, I briefly mentioned the fact that
Generic could cause silent failures at runtime if you have 2 structs that are identically shaped type-wise, but have certain fields swapped.
While we can work around this using wrapper types, that solution leaves something to be desired, because, well, more boilerplate adds noise and requires more maintenance.
Ideally, we want to have something like this, where the following works:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
but the following fails at compile-time because the fields are mis-matched (
last_name have been swapped):
1 2 3 4 5 6 7 8 9 10 11
The solution to this sort of problem has been in Shapeless for some time; by using
HLists where each cell contains not just a value, but instead hold named fields, where each value is labelled at the type level.
Let’s take a look at how Frunk implements
Field values and
LabelledGeneric in Rust :)
Have you ever wanted to convert
Hlists into Structs or to reuse logic across different types that are structurally identical or very similar (e.g. same data across different domains)?
Generic can help you do that with minimal boilerplate.
Generic is a way of representing a type in … a generic way. By coding around
Generic, you can write functions that abstract over types and arity, but still have the ability to recover your original type afterwards. This can be a fairly powerful thing.
Rust describes itself as:
a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety. ### Featuring * zero-cost abstractions * minimal runtime *efficient C bindings
So, it’s likely that developers who choose to program in Rust are focused on performance. You can make sure your code is efficient by writing benchmarks, but in order to prevent performance regressions, you’ll need to run benchmarks on your Pull Requests or patches and somehow compare before and after. Doing this can be tedious, especially as the changeset evolves over the course of code review or miscellaneous refactoring.
Let’s see how we can get automated benchmark comparisons across commits on Travis CI.
Rust comes out of the box with a
Result<T, E> type in its standard library. For those not familiar with it, it is a union-like enum type where
T is a type parameter denoting the kind object held in a
Result in the success case (
E is a type paramter denoting the kind of error object held in the failure case (
Result::Err<E>). In Scala, this is represented in the standard library as
Either[+A, +B], where the the success and error type params are swapped (traditionally, the one on the left stands for error and the one on the right is…well, right).
Result comes with really good support for what I call “early return on error”. That is, you can use
and_then (flatMap in some other languages) to transform them, and if there’s an error at an intermediate step, the chain returns early with a
1 2 3 4 5 6
But .. what happens when you have multiple
Results that are independent of each other, and you want to accumulate not only their collective success case, but also all their collective errors in the failure case?
A heterogeneous list (henceforth “HList”) is a useful abstraction that is implemented in many statically-typed functional programming languages. Unlike normal list-like structures (e.g.
Array), a heterogenous list is able to hold elements of different types (hence heterogenous) and expose those types in its own type signature.
Now, you might be thinking “Isn’t that just a tuple?”. The answer is: in a way. Indeed, in terms of data structure, a given implementation of HList is usually really nothing more than deeply nested pairs (tuple of 2 elements) that each hold an element of arbitrary type in its 1st element and knows that its 2nd element is itself an HList-like thing. While it may seem convoluted, HList buys us the ability to abstract over arity, which turns out to be extremely useful, as you can see from this Stackoverflow answer by Miles Sabin, the creater of the Shapeless library, which provides an HList implementation in Scala.
Given that description and justification for the existence of HLists, let’s take a look at how to use Frunk’s implementation of HList in Rust.
It’s been a while since the last major release of Enumeratum, and in 1.4.0, minor changes include Play 2.5 support, integration library version bumps, and small internal refactorings. More excitingly though, the latest version adds support for a new kind of enumeration,
ValueEnum, as well as an integration with the Circe JSON library.
Points of interest:
- Unlike other value enum implementations, Enumeration’s value enums perform uniqueness checks at compile time to make sure you have unique values across your enum members.
- Circe integration allows you to send and receive JSON data between your front end and your server using the same code
In Episode 1 of this series on Scala and computer vision, we created a basic Akka-Streams-powered webcam feed app. To bring it to the next level, we will dig a little deeper into the OpenCV toolset and bring in feature detection as well as video stream editing.
We will build on the foundations from the previous post and continue with the usage of Akka Streams, modeling our application as a series of small transformations that are run asynchronously, with backpressure handled automatically.
In a previous post, I talked about SBT-OpenCV, a plugin for SBT that makes it easy to get started with OpenCV in any SBT-defined JVM app using just one line in
project/plugins.sbt. Having handled the issue of getting the proper dependencies into a project, we can turn our attention to actually using the libraries to do something cool.
This post is the beginning of a series, where the end goal is to build a smile detector. Akka and OpenCV will be used, with Spark joining later on to complete the buzzwords treble.
A well-rounded and fun first step is to get a video feed from a webcam showing on our screen. To do this, we will cover a variety of things, including how to define a custom Akka
Source, how to use JavaCV, and some basic OpenCV image manipulation utilities.
OpenCV is arguably the defacto free, open-source computer vision library, but setting it up for usage in a JVM project can be hard because OpenCV itself is written in C++, so there are a bunch of system-dependent things that you need to download/compile/install before you can use it.
JavaCV, written by Bytedeco is a library that makes it more bearable to use OpenCV from JVM projects by providing a bunch of wrapper classes and logic around OpenCV (there’s a lot more to it, see their page for details).
Still, because JavaCV depends on JavaCPP for common and OpenCV C++ wrappers, and JavaCPP requires you to set your target platform (what platform you want to run on), I thought getting started could be easier still.