Cloudflare, the web infrastructure provider, is calling for increased regulation in the AI sector, citing concerns over Google’s entrenched position in search and AI markets. CEO Matthew Prince is in London meeting with the UK’s Competition and Markets Authority (CMA) to propose rules that would level the playing field for AI companies.
The CMA recently designated Google as having a “substantial and entrenched” position in search and advertising. This status allows the regulator to extend oversight beyond traditional search and ad products, covering Google’s AI Overviews, AI Mode, Discover feed, Top Stories, and News tab.
Cloudflare’s Position
Prince emphasized that Cloudflare is uniquely positioned to advise regulators because it is not an AI company but provides infrastructure for many AI businesses. “We don’t have a dog directly in the fight. We’re not an AI company,” he said, noting that 80% of AI companies are Cloudflare customers.
According to Prince, Google currently leverages its existing web crawler (Googlebot) to scrape content for AI services in addition to its search engine. This gives the tech giant a significant advantage over other AI companies like OpenAI, Anthropic, and Perplexity, who must negotiate or pay for access to the same content.
“Google is saying, ‘we have an absolute God-given right to all of the content in the world, even if we don’t pay for it.’ If you want to opt out of one, you have to opt out of both [search and AI],” Prince explained.
For media companies, blocking Google’s crawler is not practical. Doing so can cut off search traffic, which can account for around 20% of revenue, and disable Google’s ad verification tools, effectively halting advertising across platforms.
Need for a Competitive Market
Prince advocates for a market where thousands of AI companies compete to buy content from media businesses, ensuring that Google does not maintain a monopoly on access. Cloudflare has provided the CMA with detailed data showing how Google’s crawler functions and why other companies cannot easily replicate its capabilities.
Other industry leaders share similar concerns. Neil Vogel, CEO of People, Inc., which operates over 40 media brands, described Google as a “bad actor” for forcing media companies to allow crawler access in order to remain competitive. Vogel praised Cloudflare’s tools that block non-paying AI crawlers and confirmed ongoing negotiations with several major AI providers.
Conclusion
Cloudflare’s push highlights growing tension between AI development and fair competition. By advocating for stricter regulations, the company aims to ensure that Google does not continue to exploit its dual role as search engine and AI content provider, potentially shaping the rules for AI content licensing and usage in the future.
The CMA’s recognition of Google’s dominance signals that regulators are paying close attention to the interplay between AI growth and market fairness, a development that could significantly influence the AI industry landscape.