On September 28 we released 3.1 of the FP Complete public services. We’ve discussed this a bit already, but in this blog post, we want to share some of the more technical details of what we’ve added.
We’ve added a whole bunch of new features to FP Haskell Center. Here are some of the most notable:
Keep reading for a drill down of some of this list. And for some lower-level, GHC-facing bits of our work, jump ahead to the “faster” sections.
As announced previously, we’ve decided to make FP Haskell Center even more open. Since our 3.1 release, all IDE features are turned on for all users, with and without a paid account.
The new FPHC terms & conditions no longer restrict the free community version to non commercial use. Even anonymous users can access the full spread of features in the IDE, from type checking to executable generation, and even Git repositories (though you’ll definitely want to log in to be able to use that properly).
There are two distinctions between our community and paid licenses now:
Commercial licenses provide more compute power.
Under community licenses, all commits made in the IDE will be published. Similar to the GitHub model, free projects are open projects. You’ll need a commercial license for private projects, which can still be acquired by contacting FP Complete Sales. So now when you commit, you’ll see this message:
Committed changes are public
Any new commits are now made public and will make your project public. Existing private projects will not be published until you make a new commit.
NOTE: If you have an existing project with commits, it will remain private until your next commit.
Additionaly, there are now no restrictions on how you license your published code. This means that although you give others the right to view your published code, they receive no implicit license to distribute your code or use it in derived works. You can still explicitly license your code under an open source license if you choose.
Based on feedback (mainly from Edward Kmett), there’s a new directive in the code fence syntax in School of Haskell posts, which will auto-run the code snippet when the page loads. This will be especially nice for showing graphical web pages or to begin an interactive process. The code is written as:
``` haskell active autorun
main = print "Hello, World!"
```The School of Haskell now supports adding Disqus comments to your own tutorials (yet again suggested by Ed).
To enable this, go into your account settings under “Profile” and you can choose:
☑ Use site default (currently: no comments)
☐ Disable comments
☐ Use FP Complete’s Disqus account
☐ Use your own Disqus account
Disqus account ID:
[ ]
To use your own Disqus account, hit the “Use your own Disqus
account” checkbox and put your account id in the box. Then go into
your Disqus site settings under the “Advanced” tab and add
fpcomplete.com under the “Trusted Domains” text
box.
We’ve updated our default compiler version to GHC 7.8. We still provide a 7.4 environment, but it is considered deprecated, and we recommend users upgrade to 7.8. In addition, as usual, we’ve updated our library set to the newest versions of upstream packages, and will begin releasing unstable package snapshots next month.
Together with GHC 7.8 come some new compiler features. For the most part, these are simply available to you without any IDE support. We added one nifty feature though: You can now defer type errors using the checkbox in the Messages tab:
☑ Enable suggestions ☑ Enable warnings ☑ Defer type errors
This will change type errors into warnings which are instead thrown at runtime. A very useful way to keep getting type information when you have a simple type error elsewhere in the code.
This plays in very nicely with the new Type Holes feature. Now you can program Python in Haskell!
In the “Root & Filtering” tab in Settings, there is a way to
change the root directory of your project, this is similar to doing
a cd in your terminal. This is useful if you have
several projects in one repository and you want to just compile the
modules within one directory, rather than everything in the whole
repository.
You can also ignore certain files using the blacklist input,
e.g. src/Main.hs will hide that file from the list.
This can be useful for hiding things from compilation.
This feature is still considered experimental.
You should notice a definite improvement in responsive in the IDE. We’ve actually implemented many different changes to make this happen. Our network communication layer, for example, has less overhead involved, by removing some unnecessary parsing from our reverse proxy machines. We’ve also refactored quite a bit of our async architecture to provide better concurrency for backend actions.
However, the biggest change comes to how we interact with GHC. We use GHC for a few different things in the IDE:
Those first two bullets dovetail together rather nicely, though the second can take more time than the first. However, the third point doesn’t play so nicely with the other two. The issue is that, while GHC is running the bytecode, it can’t continue to compile new code. And this is definitely something we want to allow. For example, a common use case it running a web application in the IDE, while continuing to hack on it while it’s running in the background.
Before the 3.1 release, our solution was to have three copies of GHC running for each project: one to do a quick typecheck, one to extract type information, and one for running code. This meant you would get error messages quickly, but couldn’t always get type information right away. It also meant your project required much more memory, which overall slowed things down.
Our new architecture involves just a single GHC process running for each project. We load code into this process, and it typechecks, extracts information, and generates bytecode, all at the same time. The trick is what happens when you press run? We would like to use the bytecode already available in the current GHC process, without tying that process up on just running the bytecode.
The solution we ended up at is calling forkProcess.
However, it wasn’t quite as simple as that. To quote
forkProcess‘s documentation:
forkProcess comes with a giant warning: since any other running threads are not copied into the child process, it’s easy to go wrong: e.g. by accessing some shared resource that was held by another thread in the parent.
Sure enough, we ran into exactly this bug almost immediately.
And fortunately, we have some very good news to report: it’s fixed!
You can see more information in GHC trac tickets 9295 and
9296
(and check Phabricator for some
great reaction gifs). These changes have been merged upstream into
GHC. Since writing those patches, we’ve stress tested our IDE
significantly, and have no longer been able to reproduce any
forkProcess bugs, which looks incredibly
encouraging.
Note, though, that there are still some issues around
forkProcess. If you’re planning on using it in your
own code, you should be aware that you can still run into problems,
especially with multithreaded code.
One other change we made was switching to a dynamically linked GHC process. We’d wanted to do this for a while due to improved memory usage. Ultimately, we had to make the switch due to a bug in C static initializers. Now we get the nice benefit that separate processes on the same machine can share memory space for their dynamic libraries.
Subscribe to our blog via email
Email subscriptions come from our Atom feed and are handled by Blogtrottr. You will only receive notifications of blog posts, and can unsubscribe any time.