AI Leaders Confront the Internet They Helped Create

Tech executives at Anthropic, OpenAI and Google once viewed the web as an endless mine of free training data. Billions of pages, posts and articles stood ready for scraping. Fair use, they argued, justified the practice. Content creators pushed back with terms of service and robots.txt files. Few barriers held.

Now the tables have turned. Those same companies watch rivals harvest outputs from their flagship models. The technique goes by the name distillation. It lets a smaller system learn from a larger one’s responses. Spend a fraction of the compute. Replicate advanced capabilities. The irony lands hard.

The Symmetry No One Saw Coming

Anthropic sounded the loudest alarm. In February it detailed industrial-scale campaigns by Chinese labs DeepSeek, Moonshot and MiniMax. These operations used fake accounts and proxy services to query Claude at massive volume. The goal: extract reasoning, tool use and coding skills. One campaign alone generated over 16 million exchanges through 24,000 fraudulent accounts. (Anthropic)

OpenAI and Google issued similar warnings within days. They pointed to the same actors. The three firms later agreed to share intelligence via the Frontier Model Forum. The White House followed with a memo treating such extraction as a national security matter. (Business Insider)

Yet the complaints carry an echo. For years website owners watched their servers groan under waves of AI crawlers. Costs rose. Content appeared in rival products without compensation. Anthropic’s own bots, one analysis found, hit pages thousands of times for every referral sent back. The ratio stood at thousands to one. The company that positioned itself as the ethical choice led the pack in aggressive collection.

Nathan Lambert, an expert in open-source AI, labeled the outcry “distillation panic.” Researchers debate where benign self-distillation ends and competitive attacks begin. Lines blur. Some wonder if distilling public model outputs might itself qualify as fair use. The legal arguments cut both directions.

Zilan Qian, researcher at the Oxford China Policy Lab, captured the dynamic. “It’s always a kind of a cat-and-mouse game,” she told Business Insider. “As long as AI model outputs are out in the world, people will probably find a way to get access to it.”

Efforts to lock down access backfired in places. Tighter controls on top models spurred more inventive workarounds. Once signals leave the controlled environment, containment grows difficult. The internet operates on copying, remixing and redistribution. AI firms now learn this lesson firsthand.

Recent reports add layers. A July 10 analysis from the Future of Life Institute graded the industry on safety practices. Anthropic earned the top spot yet managed only a C+ overall. OpenAI and Google DeepMind followed with Cs. Every major developer weakened earlier pledges to pause development at risk thresholds. Many now tie decisions to competitors’ moves. Reviewers called it a “moving goalpost” that undermines frameworks. (BankInfoSecurity)

Sabina Nong of the Future of Life Institute put it plainly. “If we leave AI safety to corporate-defined rules, we are destined to race to the bottom.” Existential safety scored weakest across the board. No company cleared a C- in that category. Technical strategies for controlling advanced systems remain inadequate, the panel found.

But the data problem runs deeper than theft. High-quality human-generated text grows scarce. AI slop pollutes search results, social feeds and comment sections. Models trained on synthetic outputs risk inbreeding. Performance plateaus. Diversity shrinks. Experts warn of a feedback loop where machines learn from other machines. Quality suffers. Creativity narrows.

Physical and real-world data offer one path out. Robotics firms collect interaction logs from actual deployments. Humanoid platforms generate sensor streams that text corpora cannot match. Yet scaling such efforts demands infrastructure few possess. Autonomous fleets, simulators and human oversight all cost money. The frontier labs that hoovered the open web now face the limits of that approach.

Content owners adapt too. Paywalls multiply. Dynamic pages block scrapers. Some sites detect AI user-agents and serve degraded experiences. Lawsuits test the boundaries of fair use. The EU’s AI Act and U.S. proposals add regulatory pressure. No clear settlement has emerged. Negotiations between labs and publishers drag on.

So the industry stands at a crossroads. Billions poured into compute clusters. Talent wars rage. Yet the raw material for the next leap stays contested. Synthetic data helps fill gaps but introduces artifacts. Curated human feedback grows expensive. Distillation offers a shortcut that competitors exploit.

Trading Tips captured the sentiment in a recent post. “Welcome to the internet. Get used to it.” The piece highlighted how arguments once used against publishers now rebound on the model makers themselves. (Trading Tips)

Executives insist the situations differ. Training on public web text serves research. Extracting proprietary capabilities through systematic querying crosses into theft. National security enters the picture when state-linked entities join the fray. DeepSeek models appear in Chinese military procurement lists. That fact shifts the debate from copyright to geopolitics.

Still, the symmetry persists. AI companies built their empires on open information. They now scramble to close the gates. Proxies, fraud and automation make enforcement tough. Rate limits slow attackers but throttle legitimate users. Behavioral analysis helps detect suspicious patterns yet raises privacy questions. The cat keeps finding new mice.

Longer term, the fix may lie in fresh data sources. Private datasets, licensed archives and real-world telemetry could replace the public commons. Partnerships with publishers might replace scraping. Compensation models could emerge. Yet those paths demand time, money and cooperation. The current race rewards speed above all.

Anthropic’s February disclosure marked a turning point. It moved the conversation from abstract ethics to concrete business risk. Other labs followed. Governments took notice. The Frontier Model Forum now coordinates on the threat. Intelligence sharing expands. Sanctions talk grows louder.

None of this solves the underlying scarcity. Internet data, once assumed infinite and free, reveals its flaws. Noise. Bias. Obsolescence. And now, heavy contamination by machine-generated text. Models that ingest their own outputs risk collapse in coherence over generations. The inbreeding problem looms.

Researchers hunt alternatives. Multimodal signals. Video transcripts. Sensor streams from robots. Crowdsourced annotations from specialized workers. Each carries trade-offs in cost, scale and quality. Scale AI and similar firms already route labeling tasks to contractors in low-wage regions. The human element persists, hidden behind layers of outsourcing.

Industry insiders watch closely. Valuation models for AI startups hinge on sustained capability gains. If distillation levels the field, first-mover advantages erode. Billions in R&D could deliver diminishing returns. Investors ask harder questions. Safety reports like the Future of Life Institute’s add to the scrutiny.

The modern web taught everyone a lesson. Publish something and it spreads. Use it in unexpected ways. Profit from it. Complain, and others point to your own history. AI giants now live that reality. They helped shape the internet. Its rules apply to them too.

Short-term patches abound. Stronger authentication. Usage caps. Watermarking outputs. None fully stop determined actors. The economics favor extraction. Querying a live model costs far less than training from scratch. Compute constraints in certain regions make the shortcut irresistible.

So the game continues. Labs harden defenses. Attackers evolve tactics. Researchers debate definitions. Policymakers weigh intervention. And the internet keeps copying, remixing and distributing. The hard truth sinks in. No one controls information once it escapes. Not even the companies that trained on everything it offered.