Making It Easier for Websites to Express AI Training Preferences
OpenAI, Friday, October 30th, 2026
OpenAI will honor a shared robots.txt signal that lets websites state their AI training preferences to multiple AI providers at once.
OpenAI announced it will honor a common robots.txt signal that lets website operators state whether their content may be used for foundational generative AI training.
Operators can then set one preference for all participating providers instead of maintaining separate rules for each crawler.
The signal comes out of work by the IETF AI Preferences working group, and OpenAI joined other participants in announcing it.
OpenAI plans to implement it in the coming months, says its existing crawler controls will remain in place, and encouraged other AI providers to honor the signal as well.