Since 2017, lack of Haskell expertise in the core team has been a blocker to making development progress. For a while, we hoped to find a new contributor, who could help us complete the little work that remains to get the site launch ready. That hasn’t happened, and perhaps was unlikely to begin with: @Salt came across some research (I will find/add the link later) which found that, excluding developers scratching their own itch, most volunteer FLO software developers actually start off as paid contributors.
By now, it is clear that we need to change tech stacks to make progress possible again. At our team retreat last year, we decided to rewrite the frontend portion of the site (in Elm). The new site has made slow progress, which is a significant improvement over “none” in the previous 4 years. I’ve wished we could do the same for the backend, but did not think it was a viable option; too much would need to be rewritten, and the backend is already so close to complete.
Recently, though, I had an idea that I believe will sidestep (most of) the technical blockers to rewriting (most of) the backend. This post will give an overview, without going too far into the technical details.
We currently use our own account system. Following best practices for security, we do not store user passwords directly; instead we store a cryptographically secure hash, PBKDF1, using the auth library included with Yesod. PBKDF1 was a good algorithm in 2016, but is now considered out of date; before we launch, we’d like to update to something more secure.
There are two migration options:
- Discard the old hashes and require everyone to reset their password.
- When a user logs in, verify their credentials with PBKDF1; if it succeeds, create the new hash and discard the old one.
We already required a password reset once, in late 2016, and would prefer not to do so again. However, option 2 requires our new backend to be able to verify against the old hashes. There are FLO libraries for PBKDF1, but this is still some extra work; changing authentication code always requires a great amount of care.
The functionality to validate passwords already exists in the current code… so why not use that?
- Exract the validation from our current backend into a command line program
- The new backend can call it as needed.
- Wait until enough people have been migrated to the new system.
- Everyone who is a patron of Snowdrift (i.e. people who entered Credit Card info)
- Ideally the handful of most-active users here on the forum
- Throw away the old hashes, and the CLI program (done with Haskell here )
- Require everyone else to do a password reset.
So we’re not waiting forever: we already need to contact patrons anyway before we run our first charge, to confirm their credit card info. Since it has been so long for some people, we plan to let anyone who no longer wishes to be charged opt out. So, we can do double duty: anyone who has not logged in by the time we run our first charge will be dropped from being a patron; everyone who remains a patron will now be on the new system.
We’ve enabled Single Sign On for our Discourse forum (here). To keep this in place and avoid another password reset for everyone, our new backend should support this, too. Once again, it’s not very much code, but it’s still some work.
We could skip this temporarily— requiring anyone who wants to use the forum in the meantime to reset their password— and then enable it later, since if you disable and then re-enable SSO, Discourse will try to match on email instead of SSO ID (source).
In the database, we currently store pledge information and monthly donation history in terms of “Pledge Units” (1 unit = 1/10th of a cent per patron). All of our calculations, like the monthly donation and outstanding balance, are also calculated precisely in terms of Pledge Units, then converted to cents right when they are displayed on the page.
This logic isn’t very complicated (divide by 10), but it is very important; what we display on the website must be identical to what we charge. Right now, the tool for running crowdmatch calculations shares its code with the website, we don’t need to worry about them getting out of sync; I’ve been reluctant to add this potential problem.
This concern isn’t as far-fetched as it seems; it already happened once!
To simplify the presentation for users, we started displaying dollar amounts rounded to whole cents. This caused the total outstanding balance to appear inaccurate, because the calculations included tenths of a cent (really, it was the history that was inaccurate). We solved the issue by bringing back fractions of a cent… and a more complicated dashboard view.
There’s a more important reason to avoid fractions of a cent: We can’t charge them, which means patrons can’t clear their balance. It’s even worse with multiple projects, when we get there.
Instead, we should round to whole cents each month, when we calculate donations. No more tenths of a cent to display; every charge will clear the outstanding balance in full; and calculating that balance becomes dead simple (subtract charges from donations).
Solution: Make The Change
There’s a happy accident: rounding to cents each month makes our architecture more modular:
- The frontend reads from / writes to the api
- The api reads from / writes to the database
crowdmatchtool reads pledges from the database, calculates donations for that month, and store the result in the database.
- In our code base, its source is already separate from the website (thanks @chreekat!).
- It will need small updates to apply the rounding at crowdmatch-time instead of charge-time.
- A final piece will read the outstanding balance from the database, run Stripe charges in that amount, and store the result in the database.
Each of these pieces is simple, and they communicate using well-defined interfaces, making them easy to rewrite individually. This gives us a migration path that’s not “rewrite everything all at once”.
There’s another happy accident: a simple api interface like this is easy to write a spec for, which is perfect for contract work. Back to the research I mentioned at the beginning— the ideal contractor is someone who’s already heard of us and wants to volunteer, but couldn’t justify the initial time investment. This contract could double as a chance for them to get involved now, leading to both a new backend for the api and a new long-term team member.
I will write a follow-up post with the technical details of changes to the database, api, auth system.
Pun intended. Alternate title: Make It Make Cents ↩︎