I was going to start posting about this in Plan for Moving Off Haskell (Part 2): Technical Details, but I think the discussion here may get long / detailed in a way that will distract from that thread’s current focus on what changes need to happen & in what order. So here we are.
I would like the tech stack we pick to meet these criteria:
Works (aka Correctness): This should go without saying.
In-team Expertise: Our platform must be maintainable by the core team members. It’s why we’re moving off Haskell in the first place.
Boring: We already have a novel mechanism and governance structure. We don’t also need to break new ground with our tech stack.
Convenient / Approachable: We are currently all volunteers, and often have limited time. If contributing is too much of a time suck, it’s unlikely to happen. This goes double for people outside the core team; it should be relatively easy for them to contribute.
The last two points suggest that something Mainstream is desirable.
For the frontend, we’ve already chosen Elm. It’s neither Boring nor Mainstream, but @Adroit’s enthusiasm for it brings enough momentum to trump those considerations, and it is either great or good-enough on the other criteria. Plus, if for some reason we needed to migrate away, it’s relatively easy to “eject” the static content back to vanilla html + css.
Beyond that, we have decisions to make.
I know the blog I linked above cites Node as not Boring, but that was in 2015. Today, insufficiently mature alternatives are Deno or Bun. ↩︎
The most-boring option for a node api, would probably be https://expressjs.com/ + https://www.passportjs.org/, although Express and Passport are unopinionated enough that they are also not a complete stack here; they don’t come with e.g. a password reset flow built in.
As it happens, this package provides it’s own, ostensibly “production quality” client-side security + session support, utilizing jsonwebtoken for a session mechanism using signed cookies (jwt) and bcrypt for password hashing. As it’s merely a scaffold, it has no signup or reset mechanism, and uses an orm mock reading from a json file as the database.
However, things like github login, which with a passportjs + express-session combo are trivial to provide, would also need custom implementations, and I guess we needn’t reinvent the wheel…
Anyway, just surveying the landscape out loud. I’ll be around this evening for the meeting.
I looked into SuperTokens a little, and it seems like it is basically what I was originally looking for— a piece with only the auth components (password storage, password reset flow, updating email/password…) that we could plug in to our site. However, I do think it’s not quite mature enough to go with. In particular, as far as I can tell, it has no way for users to change their email. That’s a major downside, especially considering snowdrift.coop currently lacks that ability, so adding it is a major draw for switching to some off-the-shelf software.
That said, I’m glad you mentioned it, because their comparison with KeyCloak was helpful in understanding that KeyCloack is basically a more boring/mature version of the same thing. The main downsides I noticed are:
Somewhat complicated setup / bad official docs. It is mature enough that we can probably find 3rd party tutorials to get everything set up, but it’ll require some time investment and hair-pulling to figure out how to configure everything the way we want.
New addition to our tech stack: Java. This is not as big of a downside as it might be, because OSUOSL will handle deployment, so we don’t also need to get expertise in how to deploy java apps. However, any customization of the login interface (e.g. to use Snowdrift.coop branding) would need to be done in Java (I think), so it is still a significant addition to our tech stack footprint.
Overall my impression is that KeyCloak would be an acceptable solution for us to use, but not a first choice.
Small correction: You don’t need to write any Java code if all you want to do is change how the login screen / registration screen / account console / emails look. You mostly edit html, css and config files (see “creating a theme” in the docs and the source code of one of the bundled themes). You only need to write Java code if you want to change the behavior of Keycloak beyond what is configurable, e.g. if you want it to call a webhook whenever a user registers.
If you’re talking about the keycloak admin UI: There’s a lot of options, but you only need to touch some of them unless you want to do something special. You’d basically create a realm and within that realm, create a client. For both of these, you’d go through their config options and see what you need to change. Most attributes have a questionmark icon with a short description on click.
I think the most effort would be writing the custom theme, plus of course replacing the current authentication with an oauth2 client [edit 1: and importing the current users, see below]. (Assuming that OSUOSL handles deploying the instance + helps to set up a deployment pipeline for the custom theme)
An interesting question would also be what to do with the current users that are already in the database… It might be possible to configure the hashing algorithm etc sufficiently so that you can use the existing password hashes, but maybe that would take some luck
(probably became irrelevant due to edit 2)
Can you maybe create a few test users in a local instance and publish their passwords and hashes? (plus salts and algorithm if stored separately, or anything else that seems relevant) Then someone could try to get them into an instance of e.g. keycloak as a fairly well-defined task. (It’s a bit hard for me to run an instance of the snowdrift.coop site locally, last time it didn’t work because the stripe dependency needs too much RAM to compile.) I’m not promising I will do this, but maybe if I’m motivated to play around with it, I might try and report the results
Last week Two weeks ago, we discussed using Payload, which I think is our most likely option for the backend framework at this point.
I lack experience with document database schema design compared to a “traditional” RDBMS.
I think much of the crowdmatching financial data actually is a good fit for a tabular data model.
Not very mature
Small degree of lock-in, inherent to using any framework
Very little lock-in and “magic” relative to other frameworks and CMSes.
Relatively easy to “eject”— drop the framework and use access the underlying db directly —if needed.
Works well alongside custom code
If we reach framework limits, we can build what we need without scrapping everything.
Provides an incremental migration path away, if we decided we needed to drop it entirely.
We get quite a few things “for free” ⇒ fast feature development and iteration.
Auto-generated api (REST, GraphQL, and local nodejs)
Admin panel interface for us to manage data
Maybe also useful for projects to configure their project page and other things we might want to allow them to customize later.
The first 2 pros are why I don’t consider the cons to be deal-breakers. Still, I was hesitant at first, and I still have some gut reservations. And if I were only considering the migration off of Haskell, I would probably still be inclined against Payload.
However, thinking about the next steps on the road, supporting multiple projects and refining our crowdmatching mechanism, I’ve come around to the idea that a document db with a flexible schema is actually the right tool for the job.
I’m still open to being convinced that we ought to stick with postgres (or even use both?), though (I recognize that I’m blurring the lines between Payload and MongoDB here, but my strongest reservations about Payload come from it using Mongo).