A Change to Common Crawl Dataset Size Reporting | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		A Change to Common Crawl Dataset Size Reporting (commoncrawl.org)
		3 points by ccgreg 4 months ago \| hide \| past \| favorite \| 1 comment

ccgreg 4 months ago [–]

Common Crawl is switching to reporting dataset sizes in nibbles. As an organisation dedicated to data preservation, we feel it would be remiss to allow this underrepresented unit to fall out of use. Our latest crawl now exceeds 689 tebibbles. Common Crawl Foundation

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact