By Giorgio Corrias
As artificial intelligence (AI) models are becoming increasingly central in our technology-ridden lives, the question: How much do we really know about AI? Is ever so pertinent.
The answer? Less than you think.
Stanford researchers have conceived the Foundation Model Transparency Index to evaluate the transparency of foundation model developers. The index consists of 100 transparency indicators which codify transparency for foundation models, the resources required to build them, and their use in the AI models assessed. The index proceeds to score popular AI models — like OpenAI’s ChatGPT4 and Meta’s Llama 2 — on its 100 criteria, producing rankings that depict transparency, or lack thereof, within the AI ecosystem.
The indicators used are broadly divided into 3 categories: Upstream, model, and downstream. The upstream indicators refer to the computational resources, data, and labor processes involved in building a foundational model. The model indicators, rather, revolve around the properties and function – the architecture, capabilities, and risks — of the foundational model. Finally, the downstream indicators specify the distribution and use of the model, like its impact on users, the policies that control its use, and any updates to the model. The transparency index also groups indicators into subdomains, and makes further distinctions between open and closed AI models.
The highest ranked model — Meta’s Llama 2 — scored a mere 54 out of a possible 100, suggesting a fundamental lack of transparency in AI models. GPT-4 was ranked third with a transparency rating of 48%, while the mean score across the models assessed was 37%.
Rishi Bommasani, a researcher at Stanford’s Center for Research on Foundation Models (CRFM), states companies should aim for a score between 80-100, underlining a significant mismatch between the transparency required and the transparency provided by AI models. This lack of transparency makes it harder for policymakers to design policies that can rein in this powerful technology; for academics to depend upon commercial foundation models; and for businesses to safely build applications based on said models.
Percy Liang, leader at Stanford’s CRFM, labeled the project as a needed response to declining transparency within the industry. Liang asserted that “three years ago, people were publishing and releasing more details about their models… Now, there’s no information about what models these are, how they’re built and where they’re used” reports The New York Times. With more money and resources poured into AI, the largest companies constantly battle for a greater market share and dominance, resulting in increasingly widespread secrecy amongst developers.
Yet, the importance of transparency will only increase over time as models grow more powerful with millions of people incorporating AI into their lives. Hence, “it is particularly important for journalists and scientists to understand their designs, and in particular the raw ingredients, or data, that powers them” argues Shayne Longpre, PhD candidate at MIT and author of the Transparency Index.
“There are some fairly consequential decisions that are being made about the construction of these models, which are not being shared,” Liang continues.
For policymakers, transparency is essential in the designing of other policies, as these models raise fundamental questions regarding labor practices, intellectual property, and bias. In its absence, regulators lack the requisite knowledge of models, meaning they cannot pose the right questions, nor take any significant action in these areas.
For the public, transparency is imperative in allowing AI users to know what systems foundation models rely upon, so they can have a more encompassing understanding of what they are using, how to report harm induced by such systems, and how to rectify it. By permitting its users to see the extent to which the models have been tested, it enables humans to understand what is happening in AI models and acknowledge why particular decisions are made. Transparent AI is explainable AI.
Yet, transparency does not merely refer to disclosing algorithms online, as publishing lines of code – without access to the data being used – is not helpful to those who cannot make sense of it. The point is, rather, that users can explain how decisions are made by AI models. Humans must gauge the context within which an algorithm operates, so they can use their judgment to understand when a model makes a mistake, and its implications.
So, what does good communication look like in this industry?
Although no organization wants to be transparent in a manner that compromises their intellectual property, foundation models are becoming too powerful to remain this opaque. The more humans know about these systems, the more they understand their threats posed, potential benefits, and how these might be regulated by policymakers. It must be noted that transparency is not an all or nothing proposition, but companies must work to find the right balance of transparency with their stakeholders.
Identifying what their stakeholders must know in order to do their job, for instance, could be a good starting point. The Harvard Business Review makes a fitting analogy, outlining how while a model risk manager in a bank requires information regarding the model’s threshold, a Human Resources manager might need to understand how input variables are weighting in evaluating the success of interviews.
Although the information is not requisite for all stakeholders to do their jobs, if it can facilitate daily rudimentary tasks, it is a valid reason to share the information. Knowing why certain people need information is imperative in acknowledging the importance of transparency for users operating these models.
Once it is established who needs what information and why, the right kinds of explanations must be given.
For businesses, explainability fosters conditions in which technical, business, and risk professionals extract the most value from AI systems. Businesses can benefit from increasing productivity, with explainable AI more quickly revealing errors and areas for improvement while facilitating machine learning operation teams’ jobs to efficiently monitor AI systems. Additionally, if technical teams can explain the function of foundation models, business teams can affirm that an AI application is delivering its expected value with business objectives met. Explainability is also essential to building trust between users, regulators, and businesses, as well as mitigating regulatory and other risks.
As businesses increasingly rely on AI models to inform their decisions, it is imperative that users and those affected by AI understand their systems.
It is therefore essential that communications be tailored to their audiences, considering the varying levels of education and technological savviness between chief information and chief executive officers, let alone the average consumer. Hence, AI product teams must — down to the most minor of details — work with stakeholders to gauge the most lucid, effective, and intelligible way of communicating. Businesses must include explainability and transparency amongst their key principles within their responsible AI guidelines.
Though the terms explainable and transparent have been often used interchangeably in this discussion, it is important to make the distinction between the two. Explainable AI is concerned with how the model transforms its inputs into output. Its “rules,” so to speak. Transparent AI, rather, looks at everything that happened before and during the production of the model, and whether it has explainable outputs. Despite this dissimilarity, both transparency and explainability are indispensable in the pursuit of building trust between AI models and its users.
If these companies are preoccupied with disclosing trade secrets or intellectual property to rivals, they can patent their ideas, or publicly release other types of information. And, if companies are truly worried about fueling an AI revolution, they should realize by now that it is well underway.
If we are to let AI be at the center of our lives, we cannot be left in the dark by this technology and its developers.
The views expressed in this article are the author’s own, and may not reflect the opinions of The St Andrews Economist.