AI July 24, 2026 bearish ⇧ 389 pts across 1 thread

AI Scraping Is Breaking Niche Data Publishers

A thread on what happened to TheNumbers.com surfaced a pointed comment linking to a previous day's discussion where the consensus was that AI scraping complaints are usually a cover for other problems. The commenter called out the irony directly. TheNumbers.com is a specialized movie industry data site, and the implication is that aggressive AI scraping of thin-margin data publishers is a real pattern even if it gets dismissed in abstract policy debates.

This connects to a broader pattern on HN where small, specialized publishers with structured data are bearing a disproportionate cost from LLM training and inference scraping. The economics are brutal: the data is valuable precisely because it is curated, but the scraping happens at a scale that degrades the service without compensating the publisher.

One commenter suggested TheNumbers.com was a good candidate for a static site generator rewrite, which is a practical response but does not address the underlying revenue problem if the data is being consumed without driving traffic.


So what?

If your product depends on a niche third-party data source, check its health. Sites like TheNumbers.com are canaries. If you are building a data business yourself, aggressive bot blocking and rate limiting are now baseline requirements, not optional hardening. The scraping economics have changed.

Read these