The new Gatsby website has been soft launched for a couple weeks now and seems to be working pretty well so far, so I’m now moving on from building out the site to building out a bunch of tools for sending out notifications of new content. My hypothesis here is that if I can make the barrier to entry as low as possible for people to get notified of new posts, they’ll be more inclined to read them, and that means going where people already are. Obviously RSS is a great technology for this, but it never really recovered from Google Reader meeting its far too early demise. People use algorithmic news apps, social networks, chat apps, and more to find and share posts, and so that’s where I need to be.
So far I’m planning on distributing posts to:
- RSS/Atom/JSON feeds
- A Twitter account
- A Telegram channel
- Medium
- Apple News
- Mastodon
I’m a big believer in making a bunch of little systems that work together rather than one big monolithic system. I chose to build this project using the Serverless framework built on Amazon’s AWS technologies like Lambda. I really didn’t want to manage keeping processes alive and setting up things like cron jobs, so this makes a lot of sense. There are also a few other endpoints I want to implement later for the site like Micropub and Webmentions which would be nice to have managed through Lambda rather than running forever on some VPS somewhere.
Once it’s working, the basic pipeline looks something like this:
- I create a post (or make some change the site) and commit it to a Git repo
- I push the repo to my GitLab server
- When GitLab receives a commit, it kicks off a CI job to build the site and deploy it.
- Once deployed, it sends a message to an AWS Lambda function to signal the site has changed.
- The function on Lambda downloads a copy of the current website data and a copy of the last seen website data from AWS DynamoDB
- New and updated posts are sent to a WebSub server (you might remember this as “PubSubHubbub”) to signal to anyone who subscribes that the page changed. (Which means that, once this is working, the entire site will support WebSub as a server-to-server push mechanism!)
- New posts are inserted as rows into a DynamoDB table
- A series of AWS Lambda functions will be listening for insert events on that table, one for each service listed above, and they will farm out the API calls to each service
I like this approach because it’ll hopefully mean there is some resilience to normal maintenance things like server restarts and database failures. It fits nicely with a lot of open standards being pushed by the IndieWeb movement. And it’ll be automatically integrated into the build process, so I won’t forget to actually do the crossposting. It’s already well underway with a lot of the infrastructure in place; steps 1-5 and step 7 are all finished. All that’s left is to set up the individual integrations with each service and connect the WebSub plumbing.