Paramilitary Leaks

I was perusing Mastodon, as one does, and noticed a post from journalist Micah Lee saying he was exploring a 200GB dump from American paramilitary groups — a leak made possible through the incredible bravery of John Williams, who infiltrated these groups and exfiltrated the data. I hadn't heard of the dump before, and immediately sent Micah an email offering to help with any dataviz.

The leak explorer — search + a dense visual index of the document set

The pipeline chops and screws the data from Telegram HTML exports into SQLite, then Parquet hosted on Cloudflare R2 — a columnar format that's super-fast in the browser. On top of that sits a Nuxt + regl-scatterplot frontend with full-text search, filters, and a dense visual index of the corpus. We took a 200GB leak and turned it into something a reporter can explore in the browser.

Full-text search across the corpus

Explore it at para-leaks.ejfox.com, read the full write-up — Processing Telegram Leaks for Fast Web Visualization — or watch the walkthrough: Visualizing 200GB Of Leaked Telegram Chats.