I wrote a little Node application that downloads all the Cloudfront logs that Stork generates, and sticks the data in a SQLite database. I stuck a Dockerfile in front of that application. Then, I added another directory that has a Dockerfile pointing to Datasette. Now my project is a monorepo that contains multiple services.

A monorepo? Services? Complexity has skyrocketed.

I started up an EC2 box. I installed Docker on it, and I set up docker-compose and a crontab so that this box does two things: it serves my Datasette instance on port 80, and it runs the Node application on a cron job. Now, every 6 hours, my Datasette instance updates with the latest usage stats for Stork, and I can use the Datasette web instance from anywhere.

I can write SQL queries and get stats about the HTTP requests coming into the Stork CDN. It’s my own little data warehouse! Mission accomplished.

I think Observable released Plot about one day after I got Datasette working. Suddenly, I wanted—nay, needed—to do some fancy visualization with my Stork data so I can really see the stats.

I tweaked my Datasette Dockerfile to install a token-based authentication plugin. Now my data is secure, but accessible via an API. I wrote an Observable notebook that fetches the most recent usage data and plots it. Now I have a usage dashboard with a graph showing how many hits stork.js got per day this year.

It’s got a rolling average!

I reckon this is the most stable thing I’ve launched in a prod environment. This feels like the deployment with the smallest bundle of hacks I’ve ever created. I spent shockingly little time installing software on the EC2 box, which is a task I’ve easily burned weeks on before. Moreover, I can run each service independently in production or on my personal computer. I’m happy with this because I don’t think I’ll have to keep worrying about it forever.

Acknowledgements: Terin helped reaffirm that I was using Docker correctly. Thanks, Terin!