Almost a full year ago, I made the first commit into the Stackage repository, and the announcement blog post describes the project’s motivations. Since then, as part of our work at FP Complete, there’s been a lot of activity on this project. The community contributions have been amazing as well, with a huge number of package authors signing up to contribute and maintain their packages on Stackage. I want to recap where the project is right now, in terms of the package set, its own codebase, and how FP Complete is working with Stackage.
To date, users have contributed roughly 200 separate packages to the Stackage package collection. When you include all of the deep dependencies of these packages, we are currently building about 480 total packages in a full Stackage build, or roughly 9% of all Hackage packages.
This is great news for anyone using Stackage directly to build a package database, as they gain direct access to a huge number of the most popular Haskell packages, without worries of compatibility. (We’ll touch on that a bit more in the FP Complete section.) But the work we’re doing with Stackage has probably already directly benefitted all Haskell users, even those who have never touched Stackage.
As part of regular Stackage maintenance, I maintain a total of three separate daily Stackage builds, one using the most recent Haskell Platform, one using the previous release, and one using vanilla GHC 7.4 (no Haskell Platform constraints). Every day, these jobs run, try to compile all of these packages, and run the full set of test suites. The process has found a large number of simple compilation issues, overly restrictive cabal version bounds, and a number of actual bugs. In my opinion, Stackage is living up to its goal of helping maintainers raise the quality of their code.
At this point, I’ve certainly lost track of how many patches either myself or one of my coworkers at FP Complete have sent based on Stackage build results. In the cases where we haven’t been able to put a patch together, maintainers have been incredibly responsive, and I’d really like to thank everyone in the community who has participated.
So on this note, there’s only one thing I’d like to ask from people: get involved! If you’ve written some code that you’re actively maintaining, get it in Stackage. You’ll be widening the potential audience of users for your code by getting your package into FP Haskell Center, and you’ll get some helpful feedback from the automated builds so that users can more reliably build your code.
The Stackage codebase itself has become pretty stable. The initial goal was simply to try building a bunch of packages at the same time. At this point, it handles a lot more than that:
But we’re not finished yet! Security is a big concern for both FP Complete and Galois, and we’ve been talking about ways to leverage Stackage to improve the security situation of Hackage. Some concrete steps we’re hoping to take in the near future are:
Since this blog post is about the state of Stackage, and not its future, I don’t want to belabor these points too much. But expect to hear more about them in the future. In the meanwhile, if you’re interested in taking part in this process, please be in touch with me, or even better, write to the Stackage mailing list.
FP Complete is the company behind Stackage. So what are we getting out of this? Firstly, everything I’ve listed above. FP Complete is obviously very invested in the Haskell community, and benefits greatly from all of the wonderful work being done by that community. So anything we can do to help facilitate that continued progress is a boon to our progress as well. Getting a higher quality set of packages on Hackage and lowering the barrier to entry by reducing installation problems is something we’re very interested in.
More directly, many people know that we use Stackage to build the set of packages we provide on FP Haskell Center itself. When you write a School of Haskell tutorial, you have access to all 480 packages in our package database. When you start a new project in FP Haskell Center, you can immediately import any module in that package database, without concerns of diamond dependencies.
By running Stackage as such an open project, and having our system built on top of Stackage, we’ve created a natural way for users to get additional libraries onto our system. Instead of having a centralized body controlling which packages can make it onto our servers, we can open things up to a much more user-driven approach. This takes us out of the role of gate-keepers (or, said another way, bottlenecks) and instead lets us simply be enablers.
The Hackage uploader vetting has been invaluable as well. We take security very seriously, as do our users, and we need some way to ensure that when you start using package X, it hasn’t been replaced by a nefarious user. Of course, package signing will give us even more guarantees about this.
FP Complete is promoting two sets of packages currently: bleeding edge (called unstable), and the stable 13.09 package set. This latter set is based on a snapshot taken just before our first release at the beginning of this month. Using Stackage’s snapshotting and patching abilities, we are able to provide users with stable APIs from vetted packages, while fixing any bugs or security holes found in those packages on behalf of our users. And for those of you looking to be more experimental, our unstable environment can be updated much more regularly.
Stackage is fulfilling the role I’d hoped it would, both for the community in general, and at FP Complete. We’re going to be broadening its intended feature set over the next few months to include more security features, and hope to provide yet another benefit to the Haskell community. This is an exciting project to be a part of, and I hope even more of you will take part in it.
Subscribe to our blog via email
Email subscriptions come from our Atom feed and are handled by Blogtrottr. You will only receive notifications of blog posts, and can unsubscribe any time.
