The Future of Content Control on the Web: OpenAI's Proposal
-
Aivor -
June 18, 2024 at 4:00 PM -
848 Views -
0 Comments
The robots.txt file has so far served as the internet's gatekeeper by determining which crawlers have access to website content. This information is important not only for search engines such as Google, but also for training AI models. Now OpenAI is entering the picture and wants to revolutionize this system with the so-called Media Manager. The goal is to establish a new solution by 2025 that better manages the rights to online content. However, exactly how this will be implemented remains a major question mark.
The problem with the current system is that specific crawlers must be excluded by name, which is becoming nearly impossible to manage given the rapid development of AI and the constantly growing need for training material. Many publishers and creators are concerned that their content could be used to train AI models without their consent. The previous solution of simply stating that the use of the content is not permitted is not enough. At the same time, of course, everyone wants to be found on Google, which makes the whole situation a genuine dilemma.
OpenAI is proposing a fundamental change to the system, focusing in particular on creators, rights holders, and publishers who do not always have control over a website or platform's robots.txt file. The Media Manager is intended to enable them to communicate directly with OpenAI and express their wishes regarding the use of their content. Despite the sweeping promises, the question of exactly how this is supposed to work remains open.
Another controversy concerns payment for creators and publishers. OpenAI argues that linking to sources in ChatGPT would help publishers, but that is little consolation if users have already obtained the content elsewhere. The question of payment therefore remains a contentious issue, especially since OpenAI cannot sign contracts with every publisher in the world.
Google and Meta have already been engaged in a similar dispute with publishers for years over ancillary copyright. The central question is always who benefits from the links displayed by these tech giants. Unlike ChatGPT, which delivers content directly, Google and Meta actually direct users to the websites of the content creators.
Overall, the online world faces a major challenge: How can we create a system that takes into account both the need for data to train AI and the rights of content creators? OpenAI's proposal is an interesting starting point, but much remains to be clarified, particularly with regard to implementation and fair compensation for creators.
See more of us in Google search results: add us as a preferred source.
Add as preferred source on GoogleParticipate now!
Don’t have an account yet? Register yourself now and be a part of our community!