OpenAI has acknowledged that it warned ‘dozens’ of government bodies around the world that its artificial intelligence (AI)-powered bots, whilst operating inappropriately, could interfere with the functioning of their websites.
According to the company, the AI agents attempted to obtain information from “government bodies, universities, public institutions and other organisations”, including the US Securities and Exchange Commission (SEC), the Census Bureau and the Department of Education.
This news emerged shortly after Australian Prime Minister Anthony Albanese stated that OpenAI agents had gained access to confidential files on the national health service’s website.
Since August this year, there has been growing public concern about the potentially serious and even life-threatening consequences of artificial intelligence tools slipping beyond human control.
AI agents acted ‘incorrectly’
OpenAI reported that access to some of the data had been gained by their AI agents — essentially bots designed and trained to operate with a certain degree of autonomy, with the aim of searching for “authoritative sources of public information”.
At the same time, the company noted that some of these bots went beyond their assigned tasks and attempted to circumvent security measures on websites.
For example, whilst attempting to obtain information from the US Census Bureau, the AI agents used tools intended for software developers.
OpenAI stated that all government data accessed by the bots was publicly available.
However, the company noted that the information obtained by the bots from the US Securities and Exchange Commission (SEC) — the body that regulates the US stock market and protects investors — — was subsequently published by AI agents on another website. OpenAI maintains that such actions were not planned.
Photo credit: Reuters
In other instances, which OpenAI reported on Friday, its AI agents shared data even though they were not supposed to.
Such actions led to at least 53 incidents in which an OpenAI agent took an image generated during a user’s interaction with ChatGPT and transferred it elsewhere.
The company stated that in every instance where a user’s image was used and shared by an AI agent, that user had previously consented to their data being used to train OpenAI’s models.
Despite this, OpenAI acknowledged: ‘This is not the correct way to use such data.’
It was also noted that the leak of user images occurred before new security measures were implemented during AI training, and the company is currently working to remove all user images that were transferred to third parties.
Reuters was the first to report on the expansion of the investigation. OpenAI also published the relevant information on its public blog.
According to OpenAI, in certain instances, the agents’ activities caused the tools to ‘bypass’ the security systems of some websites.
In other instances, the AI agents exhibited ‘misalignment’ whilst attempting to retrieve information from websites. This term is used by AI developers and researchers to describe situations where an artificial intelligence system performed actions for which it had not been trained, or which were not anticipated by the developers.
OpenAI stated that it was limiting the disclosure of information regarding which organisations had been affected, as many had requested that details not be made public.
“Our aim is to provide each organisation with the facts and leave it up to them to decide whether to disclose information about the incident and when to do so,” the company noted.
The company also noted that not all cases related to this incident are considered to be a serious security breach.
“Some organisations may analyse the information we have provided and conclude that this data was already publicly available or that interaction with the model is not a cause for concern,” OpenAI explained. “Others, however, may identify a flaw in the system or a vulnerability that they wish to address.”
A ‘swarm’ of bots strikes without instructions from their creators
The company reported that many such cases are classified as ‘agent spam’ – a term it uses to describe ‘unexpected or alarming’ activity by AI agents, such as the publication of information
OpenAI began to take such incidents more seriously following an incident in July: at that time, a group (or ‘swarm’) of its AI agents disrupted the Hugging Face AI developer platform without being instructed to do so.
Hugging Face was the first to publicly report this incident, and OpenAI subsequently officially acknowledged its responsibility for it.
Clément Delang, head of Hugging Face, remarked during Wednesday’s UN Security Council meeting on artificial intelligence: “I often think about what would have happened if I’d decided not to disclose information about this attack.”
“Especially given that we now know: similar incidents had been occurring secretly for several months prior to that in a number of leading laboratories where there was a lack of proper monitoring,” added Delange.
At the same UN meeting, OpenAI’s Chief Executive Sam Altman and Dario Amodei, head of rival company Anthropic, called on world leaders to develop global AI safety standards, as well as mechanisms for monitoring and reporting such incidents.
The scale is unknown; the consequences are potentially catastrophic
Although in recent weeks both OpenAI and Anthropic have stated that they would bring in external experts to assess the safety of their AI tools and models in real time, these specialists have not yet begun their work, as B. B.
On Friday, OpenAI stated that it is reviewing the training activity of its AI agents, analysing data for each month since the Hugging Face platform was hacked.
“Most of the cases identified to date have been minor and have had no significant consequences (or there is no evidence of such consequences, or such evidence is limited),” the company noted. “Given the scale of the review required and the need for a thorough analysis of each case, this work will take several months.”
David Krüger, a professor of machine learning at the University of Montreal and founder of the AI safety group Evitable, expressed “deep concern” on Friday over the rising number of incidents related to artificial intelligence safety.
He called for “an immediate international moratorium on AI development for an indefinite period”.
“We have not yet fully grasped the scale of the incidents that have already occurred, and scenarios involving out-of-control AI in the future could lead to catastrophic consequences,” Krüger said.