{"$schema":"https://ejfox.com/schema/page-twin@1.json","kind":"blog_post","url":"https://ejfox.com/blog/projects/dataproofer","json_url":"https://ejfox.com/blog/projects/dataproofer.json","generator":"ejfox.com/json-twin@1","data":{"cacheVersion":"2026-05-13-gear-cards","html":"<p class=\"\">Along with Gerald Rich and the Vocativ data team, I received a <a href=\"https://knightfoundation.org/articles/20-ideas-receive-support-knight-prototype-fund-media-and-information-projects/\" target=\"_blank\" rel=\"noopener noreferrer\" class=\"external-link group inline-flex items-center text-blue-600 dark:text-blue-400\" data-preview-url=\"https://knightfoundation.org/articles/20-ideas-receive-support-knight-prototype-fund-media-and-information-projects/\">Knight Prototype Fund</a> grant to create <a href=\"https://github.com/dataproofer/Dataproofer\" target=\"_blank\" rel=\"noopener noreferrer\" class=\"external-link group inline-flex items-center text-blue-600 dark:text-blue-400\" data-preview-url=\"https://github.com/dataproofer/Dataproofer\">Dataproofer<svg class=\"inline-block w-4 h-4 ml-1 opacity-50 dark:opacity-75 group-hover:opacity-100 transition-opacity align-text-bottom\" xmlns=\"http://www.w3.org/2000/svg\" width=\"1em\" height=\"1em\" viewBox=\"0 0 24 24\"><path fill=\"currentColor\" d=\"M12 .297c-6.63 0-12 5.373-12 12c0 5.303 3.438 9.8 8.205 11.385c.6.113.82-.258.82-.577c0-.285-.01-1.04-.015-2.04c-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729c1.205.084 1.838 1.236 1.838 1.236c1.07 1.835 2.809 1.305 3.495.998c.108-.776.417-1.305.76-1.605c-2.665-.3-5.466-1.332-5.466-5.93c0-1.31.465-2.38 1.235-3.22c-.135-.303-.54-1.523.105-3.176c0 0 1.005-.322 3.3 1.23c.96-.267 1.98-.399 3-.405c1.02.006 2.04.138 3 .405c2.28-1.552 3.285-1.23 3.285-1.23c.645 1.653.24 2.873.12 3.176c.765.84 1.23 1.91 1.23 3.22c0 4.61-2.805 5.625-5.475 5.92c.42.36.81 1.096.81 2.22c0 1.606-.015 2.896-.015 3.286c0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12\" class=\"\"></path></svg></a> - an open source tool to check data for reliability, missing data, and outliers. Built as an Electron desktop app for journalists.</p>\n<figure role=\"figure\" aria-label=\"Dataproofer running its suite of data-quality tests against a CSV\" class=\"\">\n<img src=\"https://res.cloudinary.com/ejf/image/upload/c_scale,f_auto,q_auto:good,w_1280/v1780060617/projects/dataproofer/gh-4.png\" alt=\"Dataproofer running its suite of data-quality tests against a CSV\" loading=\"lazy\" decoding=\"async\" srcset=\"https://res.cloudinary.com/ejf/image/upload/c_scale,f_auto,q_auto:good,w_640/v1780060617/projects/dataproofer/gh-4.png 640w, https://res.cloudinary.com/ejf/image/upload/c_scale,f_auto,q_auto:good,w_960/v1780060617/projects/dataproofer/gh-4.png 960w, https://res.cloudinary.com/ejf/image/upload/c_scale,f_auto,q_auto:good,w_1280/v1780060617/projects/dataproofer/gh-4.png 1280w, https://res.cloudinary.com/ejf/image/upload/c_scale,f_auto,q_auto:good,w_1920/v1780060617/projects/dataproofer/gh-4.png 1920w\" sizes=\"(max-width: 430px) 92vw, (max-width: 768px) 85vw, 900px\" title=\"Dataproofer running its suite of data-quality tests against a CSV\" width=\"1200\" height=\"608\" style=\"--splay-rot:-0.6deg;\" data-dimensions=\"1200×608\" class=\"img-full my-8 rounded-sm img-splay w-full mx-auto py-4\" crossorigin=\"anonymous\">\n<figcaption class=\"\">Dataproofer running its suite of data-quality tests against a CSV</figcaption>\n</figure>\n<figure data-rehype-pretty-code-figure=\"\" class=\"\"><pre class=\"md-pre\" style=\"background-color:#09090b;color:#cccac2\" tabindex=\"0\" data-language=\"bash\" data-theme=\"\"><code data-language=\"bash\" data-theme=\"\" style=\"display: grid;\" class=\"md-codeblock\"><span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">❯</span><span style=\"color:#D5FF80\" class=\"\"> dataproofer</span><span style=\"color:#D5FF80\" class=\"\"> Criterion</span><span style=\"color:#95E6CB\" class=\"\">\\ </span><span style=\"color:#D5FF80\" class=\"\">Channel</span><span style=\"color:#95E6CB\" class=\"\">\\ </span><span style=\"color:#D5FF80\" class=\"\">videos</span><span style=\"color:#95E6CB\" class=\"\">\\ </span><span style=\"color:#D5FF80\" class=\"\">v1.2.0-20190621</span><span style=\"color:#95E6CB\" class=\"\">\\ </span><span style=\"color:#D5FF80\" class=\"\">-</span><span style=\"color:#95E6CB\" class=\"\">\\ </span><span style=\"color:#D5FF80\" class=\"\">v1.2.0-20190621.csv</span></span>\n<span class=\"\" data-line=\"\"> </span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">total</span><span style=\"color:#D5FF80\" class=\"\"> rows</span><span style=\"color:#DFBFFF\" class=\"\"> 5035</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">rows</span><span style=\"color:#D5FF80\" class=\"\"> sampled</span><span style=\"color:#DFBFFF\" class=\"\"> 1259</span></span>\n<span class=\"\" data-line=\"\"> </span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Missing</span><span style=\"color:#D5FF80\" class=\"\"> or</span><span style=\"color:#D5FF80\" class=\"\"> duplicate</span><span style=\"color:#D5FF80\" class=\"\"> column</span><span style=\"color:#D5FF80\" class=\"\"> headers:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Numeric</span><span style=\"color:#D5FF80\" class=\"\"> cells:</span><span style=\"color:#D5FF80\" class=\"\"> info</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">String</span><span style=\"color:#D5FF80\" class=\"\"> cells:</span><span style=\"color:#D5FF80\" class=\"\"> info</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Empty</span><span style=\"color:#D5FF80\" class=\"\"> Cells:</span><span style=\"color:#D5FF80\" class=\"\"> warn</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Special</span><span style=\"color:#D5FF80\" class=\"\"> Letters</span><span style=\"color:#CCCAC2B3\" class=\"\"> &#x26;</span><span style=\"color:#73D0FF\" class=\"\"> Characters:</span><span style=\"color:#D5FF80\" class=\"\"> warn</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Duplicate</span><span style=\"color:#D5FF80\" class=\"\"> Rows:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Potentially</span><span style=\"color:#D5FF80\" class=\"\"> missing</span><span style=\"color:#D5FF80\" class=\"\"> rows:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Words</span><span style=\"color:#D5FF80\" class=\"\"> at</span><span style=\"color:#D5FF80\" class=\"\"> their</span><span style=\"color:#D5FF80\" class=\"\"> character</span><span style=\"color:#D5FF80\" class=\"\"> limit:</span><span style=\"color:#D5FF80\" class=\"\"> failed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Integer</span><span style=\"color:#D5FF80\" class=\"\"> at</span><span style=\"color:#D5FF80\" class=\"\"> its</span><span style=\"color:#D5FF80\" class=\"\"> SQL</span><span style=\"color:#D5FF80\" class=\"\"> upper</span><span style=\"color:#D5FF80\" class=\"\"> limit:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Summed</span><span style=\"color:#D5FF80\" class=\"\"> integer</span><span style=\"color:#D5FF80\" class=\"\"> at</span><span style=\"color:#D5FF80\" class=\"\"> its</span><span style=\"color:#D5FF80\" class=\"\"> upper</span><span style=\"color:#D5FF80\" class=\"\"> limit:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Small</span><span style=\"color:#D5FF80\" class=\"\"> integer</span><span style=\"color:#D5FF80\" class=\"\"> at</span><span style=\"color:#D5FF80\" class=\"\"> its</span><span style=\"color:#D5FF80\" class=\"\"> SQL</span><span style=\"color:#D5FF80\" class=\"\"> upper</span><span style=\"color:#D5FF80\" class=\"\"> limit:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Big</span><span style=\"color:#D5FF80\" class=\"\"> integer</span><span style=\"color:#D5FF80\" class=\"\"> at</span><span style=\"color:#D5FF80\" class=\"\"> its</span><span style=\"color:#D5FF80\" class=\"\"> SQL</span><span style=\"color:#D5FF80\" class=\"\"> upper</span><span style=\"color:#D5FF80\" class=\"\"> limit:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Outliers</span><span style=\"color:#D5FF80\" class=\"\"> from</span><span style=\"color:#D5FF80\" class=\"\"> the</span><span style=\"color:#D5FF80\" class=\"\"> mean:</span><span style=\"color:#D5FF80\" class=\"\"> info</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Outliers</span><span style=\"color:#D5FF80\" class=\"\"> from</span><span style=\"color:#D5FF80\" class=\"\"> the</span><span style=\"color:#D5FF80\" class=\"\"> median:</span><span style=\"color:#D5FF80\" class=\"\"> info</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Invalid</span><span style=\"color:#D5FF80\" class=\"\"> coordinates:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">Void</span><span style=\"color:#D5FF80\" class=\"\"> coordinates:</span><span style=\"color:#D5FF80\" class=\"\"> passed</span></span>\n<span class=\"\" data-line=\"\"> </span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">75%</span></span>\n<span class=\"\" data-line=\"\"><span style=\"color:#73D0FF\" class=\"\">9</span><span style=\"color:#D5FF80\" class=\"\"> tests</span><span style=\"color:#D5FF80\" class=\"\"> passed</span><span style=\"color:#D5FF80\" class=\"\"> out</span><span style=\"color:#D5FF80\" class=\"\"> of</span><span style=\"color:#DFBFFF\" class=\"\"> 12</span></span>\n<span class=\"\" data-line=\"\"> </span>\n<span class=\"\" data-line=\"\"><span style=\"color:#B8CFE680;font-style:italic\" class=\"\">### PROOFED ###</span></span><button type=\"button\" title=\"Copy code\" aria-label=\"Copy code\" data=\"❯ dataproofer Criterion\\ Channel\\ videos\\ v1.2.0-20190621\\ -\\ v1.2.0-20190621.csv\n\ntotal rows 5035\nrows sampled 1259\n\nMissing or duplicate column headers: passed\nNumeric cells: info\nString cells: info\nEmpty Cells: warn\nSpecial Letters &#x26; Characters: warn\nDuplicate Rows: passed\nPotentially missing rows: passed\nWords at their character limit: failed\nInteger at its SQL upper limit: passed\nSummed integer at its upper limit: passed\nSmall integer at its SQL upper limit: passed\nBig integer at its SQL upper limit: passed\nOutliers from the mean: info\nOutliers from the median: info\nInvalid coordinates: passed\nVoid coordinates: passed\n\n75%\n9 tests passed out of 12\n\n### PROOFED ###\" class=\"rehype-pretty-copy\" onclick=\"navigator.clipboard.writeText(this.attributes.data.value);this.classList.add(&#x27;rehype-pretty-copied&#x27;);window.setTimeout(() => this.classList.remove(&#x27;rehype-pretty-copied&#x27;), 3000);\"><span class=\"ready\"></span><span class=\"success\"></span></button><style class=\"\">:root {--copy-icon: url(\"data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 48 48'%3E%3Cpath fill='%23adadad' d='M16.187 9.5H12.25a1.75 1.75 0 0 0-1.75 1.75v28.5c0 .967.784 1.75 1.75 1.75h23.5a1.75 1.75 0 0 0 1.75-1.75v-28.5a1.75 1.75 0 0 0-1.75-1.75h-3.937a4.25 4.25 0 0 1-4.063 3h-7.5a4.25 4.25 0 0 1-4.063-3M31.813 7h3.937A4.25 4.25 0 0 1 40 11.25v28.5A4.25 4.25 0 0 1 35.75 44h-23.5A4.25 4.25 0 0 1 8 39.75v-28.5A4.25 4.25 0 0 1 12.25 7h3.937a4.25 4.25 0 0 1 4.063-3h7.5a4.25 4.25 0 0 1 4.063 3M18.5 8.25c0 .966.784 1.75 1.75 1.75h7.5a1.75 1.75 0 1 0 0-3.5h-7.5a1.75 1.75 0 0 0-1.75 1.75'/%3E%3C/svg%3E\");--success-icon: url(\"data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath fill='%2366ff85' d='M9 16.17L5.53 12.7a.996.996 0 1 0-1.41 1.41l4.18 4.18c.39.39 1.02.39 1.41 0L20.29 7.71a.996.996 0 1 0-1.41-1.41z'/%3E%3C/svg%3E\");}pre:has(code) {position: relative;}pre button.rehype-pretty-copy {right: 1px;padding: 0;width: 24px;height: 24px;display: flex;margin-top: 2px;margin-right: 8px;position: absolute;border-radius: 25%;backdrop-filter: blur(3px);& span {width: 100%;aspect-ratio: 1 / 1;}& .ready {background-image: var(--copy-icon);}& .success {display: none; background-image: var(--success-icon);}}&.rehype-pretty-copied {& .success {display: block;} & .ready {display: none;}}pre button.rehype-pretty-copy.rehype-pretty-copied {opacity: 1;& .ready { display: none; }& .success { display: block; }}</style></code></pre></figure>","title":"Dataproofer","metadata":{"title":"Dataproofer","date":"2015-11-03T05:00:00.000Z","modified":"2025-08-26T19:53:33.000Z","url":"https://github.com/dataproofer/Dataproofer","tech":["JavaScript","Electron","Data Quality","CSV"],"featured":false,"state":"deployed","aiInvolvement":"human-only","context":"collaborative","tags":["data","javascript","tool","journalism"],"words":157,"images":1,"imageDetails":{"total":1,"cloudinary":1,"withDimensions":0},"links":3,"codeBlocks":1,"headers":{},"toc":[],"type":"post"}},"_links":{"self":"https://ejfox.com/blog/projects/dataproofer.json","html":"https://ejfox.com/blog/projects/dataproofer","index":"/blog.json"}}