Web infrastructure giant Cloudflare just made a major change to automatically block AI crawlers by default on new websites, alongside the launch of a marketplace where publishers can charge bots micropayments for accessing content.
• Cloudflare will require AI companies to get explicit permission before scraping any of the 20% of websites it protects, reversing decades of open web policies.
• Publishers can set individual prices for AI crawlers through Pay per Crawl, choosing whether bots pay for training data, search results, or other uses.
• Media outlets like Condé Nast, TIME, and The Atlantic joined the initiative, citing traffic losses due to AI answering queries without the original sources.
This potentially positions Cloudflare as one of the gatekeepers for the data needed for a coming wave of agents that browse on our behalf. The marketplace could force healthier AI-publisher relationships, but also might create an internet divided between premium content and free sites that become AI's default sources.
OpenAI is building out a consulting arm that charges enterprises at least $10M to customize AI models, according to a new report from The Information — putting the AI leader in competition with industry giants like Palantir and Accenture.
Customers must commit at least $10M for access to OpenAI researchers, with some deals reaching hundreds of millions over multiple years.
Companies are rushing to integrate AI, and who better to guide them than the researchers who helped build it? Custom AI models trained on proprietary data unlock capabilities that generic chatbots can't touch and could result in billions in operational efficiency and competitive advantage — even with the lofty price tags.