Plan for Moving Off Haskell (Part 1): Overview

Context: We Need to Move Off Haskell

Since 2017, lack of Haskell expertise in the core team has been a blocker to making development progress. For a while, we hoped to find a new contributor, who could help us complete the little work that remains to get the site launch ready. That hasn’t happened, and perhaps was unlikely to begin with: @Salt came across some research (I will find/add the link later) which found that, excluding developers scratching their own itch, most volunteer FLO software developers actually start off as paid contributors.

By now, it is clear that we need to change tech stacks to make progress possible again. At our team retreat last year, we decided to rewrite the frontend portion of the site (in Elm). The new site has made slow progress, which is a significant improvement over “none” in the previous 4 years. I’ve wished we could do the same for the backend, but did not think it was a viable option; too much would need to be rewritten, and the backend is already so close to complete.

Recently, though, I had an idea that I believe will sidestep (most of) the technical blockers to rewriting (most of) the backend. This post will give an overview, without going too far into the technical details.

Blocker #1: Auth

We currently use our own account system. Following best practices for security, we do not store user passwords directly; instead we store a cryptographically secure hash, PBKDF1, using the auth library included with Yesod. PBKDF1 was a good algorithm in 2016, but is now considered out of date; before we launch, we’d like to update to something more secure.

There are two migration options:

  1. Discard the old hashes and require everyone to reset their password.
  2. When a user logs in, verify their credentials with PBKDF1; if it succeeds, create the new hash and discard the old one.

We already required a password reset once, in late 2016, and would prefer not to do so again. However, option 2 requires our new backend to be able to verify against the old hashes. There are FLO libraries for PBKDF1, but this is still some extra work; changing authentication code always requires a great amount of care.

Solution: CLI Validation

The functionality to validate passwords already exists in the current code… so why not use that?

  • Exract the validation from our current backend into a command line program
  • The new backend can call it as needed.
  • Wait until enough people have been migrated to the new system.
    • Everyone who is a patron of Snowdrift (i.e. people who entered Credit Card info)
    • Ideally the handful of most-active users here on the forum
  • Throw away the old hashes, and the CLI program (done with Haskell here :tada:)
  • Require everyone else to do a password reset.

So we’re not waiting forever: we already need to contact patrons anyway before we run our first charge, to confirm their credit card info. Since it has been so long for some people, we plan to let anyone who no longer wishes to be charged opt out. So, we can do double duty: anyone who has not logged in by the time we run our first charge will be dropped from being a patron; everyone who remains a patron will now be on the new system.

Discourse Single-Sign-On

We’ve enabled Single Sign On for our Discourse forum (here). To keep this in place and avoid another password reset for everyone, our new backend should support this, too. Once again, it’s not very much code, but it’s still some work.

We could skip this temporarily— requiring anyone who wants to use the forum in the meantime to reset their password— and then enable it later, since if you disable and then re-enable SSO, Discourse will try to match on email instead of SSO ID (source).

Blocker #2: Money calculations

In the database, we currently store pledge information and monthly donation history in terms of “Pledge Units” (1 unit = 1/10th of a cent per patron). All of our calculations, like the monthly donation and outstanding balance, are also calculated precisely in terms of Pledge Units, then converted to cents right when they are displayed on the page.

This logic isn’t very complicated (divide by 10), but it is very important; what we display on the website must be identical to what we charge. Right now, the tool for running crowdmatch calculations shares its code with the website, we don’t need to worry about them getting out of sync; I’ve been reluctant to add this potential problem.

This concern isn’t as far-fetched as it seems; it already happened once!

Fractional Cent Problems

To simplify the presentation for users, we started displaying dollar amounts rounded to whole cents. This caused the total outstanding balance to appear inaccurate, because the calculations included tenths of a cent (really, it was the history that was inaccurate). We solved the issue by bringing back fractions of a cent… and a more complicated dashboard view.

image

There’s a more important reason to avoid fractions of a cent: We can’t charge them, which means patrons can’t clear their balance. It’s even worse with multiple projects, when we get there.

Instead, we should round to whole cents each month, when we calculate donations. No more tenths of a cent to display; every charge will clear the outstanding balance in full; and calculating that balance becomes dead simple (subtract charges from donations).

Solution: Make The Change[1]

There’s a happy accident: rounding to cents each month makes our architecture more modular:

  • The frontend reads from / writes to the api
  • The api reads from / writes to the database
  • The crowdmatch tool reads pledges from the database, calculates donations for that month, and store the result in the database.
    • In our code base, its source is already separate from the website (thanks @chreekat!).
    • It will need small updates to apply the rounding at crowdmatch-time instead of charge-time.
  • A final piece will read the outstanding balance from the database, run Stripe charges in that amount, and store the result in the database.

Each of these pieces is simple, and they communicate using well-defined interfaces, making them easy to rewrite individually. This gives us a migration path that’s not “rewrite everything all at once”.

Strategy: Put Out a Contract For This Work

There’s another happy accident: a simple api interface like this is easy to write a spec for, which is perfect for contract work. Back to the research I mentioned at the beginning— the ideal contractor is someone who’s already heard of us and wants to volunteer, but couldn’t justify the initial time investment. This contract could double as a chance for them to get involved now, leading to both a new backend for the api and a new long-term team member.

Follow-up post

I will write a follow-up post with the technical details of changes to the database, api, auth system.


  1. Pun intended. Alternate title: Make It Make Cents ↩︎

3 Appreciations

More on Auth

I realized I missed an important point above: a big reason to move to another auth system is to avoid wasting time on generic features outside our core offering. Two parts:

Features we don’t support right now

There’s a lot of auth system features that we’d like but don’t have right now.

  • Changing your email
  • 2fa

And some which are less important or maybe we don’t want them right now, but might be nice to add in the future:

  • Using multiple emails
  • OpenID / federated sign in
  • OAuth workflows (for connecting other Snowdrift services, like the wiki, with our login system)

Fixing Our Password Reset Flow

Right now we follow a non-standard password reset flow, which is worse than the standard one.

Standard password reset flow

  • Click “Reset Password” → Enter your email
  • Follow link from email → Enter your new password

Our password reset flow

  • Click “Reset Password” → Enter your email and new password
  • Follow link from email → New password is confirmed

On the happy path, the two flows are pretty much equivalent. However, ours runs into problems in more complicated scenarios:

  • With a delay: After sending the reset, you either have to update your password manager right away (now it’s wrong until you click the link) or remember what password you entered when you initiated the reset (you might not remember if you activate it a day or two later).

  • Resetting twice: If you send two resets, and use a different password each time, you have to keep track of which password you actually activated.

  • Phishing: If a malicious person wanted to get access to your account, they can send a password reset attempt to your email with a password of their choice. All they have to do is trick you into clicking the confirmation link. In the standard flow, you know exactly what you’re changing your password to, because it’s updated immediately after you type it.

Also generally doing things the standard way is less confusing, it’s one less thing for people to remember.


Adding all this (and making sure it’s done right) is a significant amount of work, that would be better spent on e.g. getting payments functioning.