A look inside the indictment accusing New York City's mayor of taking bribes
Australia and China agree to cooperate further on investment, trade
A crucial party election Friday will determine Japan's new prime minister
Top climate negotiators urge tougher targets, but outsiders say they aren't cutting back themselves
Japan’s next prime minister could be its first woman leader
China central bank cuts seven-day reverse repo rate to aid economy
Oil prices fall as expectations of higher supplies hammer market sentiment
China cuts banks' reserve ratio as economic growth sputters
Yankees win AL East title with 10-1 victory over Orioles behind Judge, Stanton and Cole
Demi Lovato joins Gavin Newsom as he signs legislation protecting young social media influencers
‘Today there is crying in baseball’: Athletics bid farewell to Oakland with one final victory
By Anna Tong
(Reuters) - Stanford University researchers issued a report on Wednesday measuring the transparency of artificial intelligence foundation models from companies like OpenAI and Google, and the authors urged the companies to reveal more information such as the data and human labor used to train models.
“It is clear over the last three years that transparency is on the decline while capability is going through the roof," said Stanford professor Percy Liang, a researcher behind the Foundation Model Transparency Index. “We view this as highly problematic because we've seen in other areas like social media that when transparency goes down, bad things can happen as a result."
Foundation models are AI systems trained on massive datasets that can perform a variety of tasks from writing to coding. Companies that develop foundation models are driving the surge in generative AI, which since the launch of Microsoft-backed OpenAI's hit product ChatGPT has captivated businesses of all sizes.
In a world increasingly relying on these models for decision-making and automation, understanding their limitations and biases is crucial, the report’s authors say.
The index graded 10 popular models on 100 different transparency indicators, such as training data and how much compute was used. All models scored "unimpressively": even the most transparent model, Meta’s Llama 2, received a score of 53 out of 100. Amazon’s Titan model ranked the lowest, at 11 out of 100. OpenAI’s GPT-4 model received a score of 47 out of 100.
The index’s authors hope that the report will encourage companies to increase their foundation model transparency, and also serve as a starting point for governments grappling with how to regulate the burgeoning field.
The index is a project from the Stanford Institute for Human-Centered Artificial Intelligence’s Center for Research on Foundation Models.
(Reporting by Anna Tong in San Francisco; Editing by David Gregorio)
