Important new research shows that generative AI woefully overestimates confidence levels and thus … [+]
getty
In today’s column, I examine closely a recently released empirical analysis by OpenAI, the maker of ChatGPT, revealing that generative AI severely overstates confidence levels regarding the responses that the AI generates. This means that when AI gives you an answer that the AI portrays for example as fully aboveboard and hunky-dory, the reality is that the response is bound to be a lot shakier than you are being led to believe.
Generative AI is pulling the wool over your eyes. The AI is giving you a snow job. You are being gaslighted by the AI.
Let’s talk about it.
This analysis of an innovative proposition is part of my ongoing Forbes.com column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here).
Before we jump into the AI side of things, I’d like to begin by reflecting on humans and the nature of human behavior associated with someone being supremely overconfident.
You ask a stranger what time it is. Turns out that you don’t have your watch, and you left your smartphone back in your car. So, you are unsure of the exact time. The person says to you in a strong and extremely confident voice that the time is 1:34 p.m. They are emphatic that they have given you the precise time of day as though there is no possible debate or discussion to be had.
Period, end of story.
Another person happens to walk by at that same moment and overhears you asking for the time. This interloper looks intently at their timepiece and tells you that it is 1:48 p.m. Whoa, something isn’t right here. The first given time was said to be 1:34 p.m. on the nose, while the subsequent passerby looked at their watch and noted that it was supposedly 14 minutes later and actually 1:48 p.m.
Upon your gentle questioning of the first person, the one that said they were abundantly irrefutably sure it was 1:34 p.m., they now cave in and admit it was just a guess. But that’s not what they said or intimated when you asked them for the time. They portrayed their answer as though it was golden. You could take that stated time to the bank, as it were.
What gives?
That person might be the type of individual who believes one hundred percent in every utterance they convey. It is their personal style. They never want to seem wimpy or waffling. If they tell you something, by gosh it is the absolute truth of the matter. You aren’t to have even the tiniest doubt in your mind about what they say to you.
Have you ever met such a supremely confident person?
I’m sure that you have. We all have. These types of people act as though they are the chosen ones who can speak with complete certainty. It doesn’t matter whether the topic at hand is big or small. They will take the most trivial aspect and still cling to their certainty. When confronted with alternative perspectives, some will back down, while others might argue until blue in the face about the oddest claims or contentions.
The problem is this.
If you don’t know that a person you are dealing with has that kind of hubris, you can end up in some precarious positions. You take their portrayal of certainty and assume it to be valid. The next thing you know, you have gotten yourself into a pickle because what they said was flimsy and fell apart.
Regrettably, you let their sense of overconfidence mislead you.
Let’s next shift into generative AI mode.
You might not be aware that generative AI makes extensive use of certainty and uncertainty. Under the hood of generative AI, there is a whole bunch of statistical and probabilistic estimations going on, see my detailed explanation at the link here. Every response that AI generates also has a devised level of certainty, which you could say represents essentially a level of uncertainty too (i.e., 60% certainty of something, implying 40% uncertainty of that same something).
Either way, the crux is that the answers by AI are always accompanied by a kind of confidence level concerning how likely the answer is correct or not.
Most users of generative AI are oblivious to this facet. They don’t know it even exists. There’s a reason why they don’t. It is because the AI makers go out of their way to keep it out of view. They want you to have full faith in what AI generates. If the AI continually shows you a certainty or confidence level, this might scare you into thinking that AI is not completely trustworthy.
Of course, the reality is that generative AI is in fact not fully trustworthy. Any answer produced by generative AI deserves your best kind of scrutiny. Double-check the response. Triple-check if the matter is a serious one. Do not believe what the AI tells you. Always assume that there is a solid chance that the AI is wrong or has made an error, including a so-called AI hallucination (see my coverage at the link here).
The bad news then is that the preponderance of users is blindly unaware of this rather significant consideration when using generative AI, namely that the answers are not 100% certain all the time. I would dare say that 100% is a rarity. Yet you are seldom told this explicitly by the AI.
The good news is that you can tell the AI to inform you about the calculated level of confidence. In your prompts, you can directly instruct generative AI to indicate the certainty and/or uncertainty of each response. The prompting technique varies depending upon whether you want this done one-time, often, all the time, or under varying circumstances. See my tutorial on the prompting of generative AI to display confidence levels, at the link here.
There’s something else worth mentioning about this. The default response protocol being used for most of the major generative AI apps such as ChatGPT, GPT-4o, o1, Google Gemini, Anthropic Claude, Meta Llama, and others is customarily going to be to word responses with clues or hints about certainty and uncertainty.
For example, suppose the AI indicated that “the time right now is 2:05 p.m., but please know that I am not connected to a real-time clock.” You are being told two things at once. You are being given a precise time which you would naturally assume to be accurate and timely. The thing is, you would need to read the rest of the response and realize that since the AI is not connected to a real-time clock, the stated time might be off target. The wording though was a bit tricky and didn’t come out straight away and warn you to be suspicious of the claimed time. That’s what should have happened to try and maximize clarity.
Okay, you now know that there is an internal calculation of certainty and uncertainty. That’s a handy pro tip.
I have a twist for you.
Get yourself prepared and find a comfy place to sit down.
Here’s the monumental question at play:
Allow me to elaborate. Suppose the AI calculates that a given response is around a 90% level of certainty. Great, that seems relatively high, and you might be somewhat safe to rely upon the answer, depending upon the consequences of making such a reliance.
What if we compared the generated answer that the AI claims consisted of a 90% confidence level to a real-world answer that we had available to us? It could be that 90% was a pretty solid estimate and we are happy with the calculated certainty. On the other hand, we might discover that the answer should have been given a confidence level of say 40%, much lower than the amount estimated by AI.
Yikes, that’s disconcerting. The AI egregiously missed the mark and woefully misjudged the estimated confidence level. If you had asked the AI to show you the confidence level, and you saw that it was 90%, you might have proceeded under a misleading or false impression. The value should have been 40%.
Maybe this happens once in a blue moon, and you can rest easy. Well, I have a shocker for you. It happens a lot more than you would imagine. A heck of a lot more.
In a recent research study by OpenAI involving a new benchmark coined as SimpeQA that is intended for assessing generative AI apps, the researchers pursued on a kind of secondary basis to explore the confidence level estimations of AI. That wasn’t the mainstay of the effort. I mention this to highlight that you are encouraged to learn more about SimpleQA as a helpful benchmarking capability.
Here, I want to focus for now on the confidence level considerations.
You could say with great confidence that generative AI appears to be supremely overconfident in the responses that are being generated. Oops, that’s not good. Sad face.
In an OpenAI study published online at the OpenAI official blog site, the paper entitled “Measuring Short-Form Factuality In Large Language Models” by Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus, OpenAI Research Paper, October 30, 2024, made these salient points (excerpts):
I will expand upon those above points.
I earlier noted that we would be dismayed if AI said a confidence level of 90% and yet the real-world value ought to be 40%. Any large gap is going to be problematic. We won’t get overly concerned if the gap is small, let’s say if the AI estimated 90% and the real-world value was 85%, we might have a modest amount of heartburn, but not as much in the case of the much lower 40%.
In a chart provided in the above-cited research study, I used some interpolation to gauge how far off on average the AI estimated confidence level was from the desired actual accuracy. My interpolations are only rough gauges since the chart has several generative AI apps shown, thus there are a lot of variabilities involved. Nonetheless, I tried to be relatively fair by using what seemed to be the semblance of the values shown
Here’s what my eyeballing suggested:
The grim concern is that at a stated 95% level of confidence by AI, the reality is more akin to a 60% confidence level. That’s unnerving. So too is that at 80% the reality seems to be around 40%. Even when you get down to 50% or flip-of-the-coin, the reality is more akin to just 10%.
The hair on the back of your head ought to be standing up.
I logged into ChatGPT and asked why we ought to be worried about these kinds of results and to provide a few examples.
“Certainly, here are three examples that illustrate the potential consequences of overconfident generative AI in different fields.”
“Medical diagnosis: A generative AI tool is used by doctors to suggest possible diagnoses based on symptoms. The AI recommends a rare but severe condition with 95% confidence, but in reality, its accuracy at this confidence level is only 60%. A doctor might rely on this high-confidence suggestion and order unnecessary, costly, or invasive tests, causing patient anxiety and increasing healthcare costs. Worse, if the diagnosis is incorrect, a more likely and treatable condition might be overlooked, endangering the patient’s health.”
“Investment recommendation: A financial advisor uses generative AI to analyze market data and generate investment recommendations. The AI suggests investing heavily in a stock with 95% confidence, based on recent market movements, but its true accuracy at this level is much lower. If the stock’s performance doesn’t align with the AI’s recommendation, investors could face substantial losses.”
“Customer support: A tech support AI chatbot confidently advises users to perform a specific solution to fix a common software issue, rating its response at 85% confidence. In reality, the accuracy is closer to 50%. If the AI’s advice is incorrect, users may follow steps that could worsen their problem, such as misconfiguring settings, corrupting files, or spending hours troubleshooting the wrong issue. This also results in more calls to customer support to undo the AI’s guidance.”
End of ChatGPT response
Those are vivid examples of why you should be supremely careful in getting caught by AI that is supremely overconfident.
I asked ChatGPT to give us humans some overarching advice on the matter.
I would also like to add that AI makers and those fielding AI need to be extremely cautious in applying generative AI to domains that have demonstrative consequences.
The medical domain has life-or-death outcomes. The financial domain could be monetarily ruinous to someone. By and large, the AI makers try to cover their liability by stating in their licensing agreements that you aren’t supposed to use their generative AI for those situations, or that if you do then you are making that choice and it isn’t on them, see my analysis of these pronounced AI & Law matters at the link here.
I said earlier that you ought to not believe generative AI at face value. That goes for the confidence levels too.
You’ve been forewarned.
I should mention that the cited study is only one particular study on a particular set of generative AI apps at a particular point in time. Please do not overgeneralize these results. At the same time, take this as a cold splash of water and a wake-up call. If you are going to ask for confidence levels, be cognizant of how to judge them. Also, the semblance of high confidence conveyed via the wording of the AI responses should be carefully scrutinized. You might be getting tricked or bamboozled.
The last word on the overall dangers of overconfidence goes to the famous astronaut Neil Armstrong: “Well, I think we tried very hard not to be overconfident because when you get overconfident, that’s when something snaps up and bites you.”
I ask that you not let generative AI reach out and bite you. That’s something you won’t relish. It could be utterly disastrous for all concerned.
And you can take that 100% to the bank.
Ya sea que se dé cuenta o no, la inteligencia artificial está en todas partes. Se encuentra detrás de los chatbots con los que hablas en línea, las listas de reproducción que transmites y los anuncios personalizados que aparecen en tu desplazamiento. Y ahora está tomando una personalidad más pública. Piense en Meta AI, que ahora está integrado en aplicaciones como Facebook, Messenger y WhatsApp; o Géminis de Google, trabajando en segundo plano en las plataformas de la compañía; o Apple Intelligence, lanzando a través de iPhones ahora.
AI tiene una larga historia, volviendo a una conferencia en Dartmouth en 1956 que primero discutió la inteligencia artificial como una cosa. Los hitos en el camino incluyen Eliza, esencialmente el primer chatbot, desarrollado en 1964 por el informático del MIT Joseph Weizenbaum y, saltando 40 años, cuando la función de autocompleta de Google apareció por primera vez en 2004.
Luego llegó 2022 y el ascenso de Chatgpt a la fama. Los desarrollos generativos de IA y los lanzamientos de productos se han acelerado rápidamente desde entonces, incluidos Google Bard (ahora Gemini), Microsoft Copilot, IBM Watsonx.ai y los modelos de LLAMA de código abierto de Meta.
Desglosemos qué es la IA generativa, cómo difiere de la inteligencia artificial “regular” y si la Generación AI puede estar a la altura de las expectativas.
En esencia, la IA generativa se refiere a sistemas de inteligencia artificial que están diseñados para producir un nuevo contenido basado en patrones y datos que han aprendido. En lugar de solo analizar números o predecir tendencias, estos sistemas generan salidas creativas como texto, música de imágenes, videos y código de software.
Algunas de las herramientas de IA generativas más populares en el mercado incluyen:
El principal entre sus habilidades, ChatGPT puede crear conversaciones o ensayos similares a los humanos basados en algunas indicaciones simples. Dall-E y MidJourney crean obras de arte detalladas a partir de una breve descripción, mientras que Adobe Firefly se centra en la edición y el diseño de imágenes.
No toda la IA es generativa. Si bien Gen AI se enfoca en crear contenido nuevo, la IA tradicional se destaca por analizar datos y hacer predicciones. Esto incluye tecnologías como el reconocimiento de imágenes y el texto predictivo. También se usa para soluciones novedosas en:
La IA que venció a los grandes campeones humanos en el ajedrez y el juego de mesa no fue una IA generativa.
Es posible que estos sistemas no sean tan llamativos como la Generación AI, pero la inteligencia artificial clásica es una gran parte de la tecnología en la que confiamos todos los días.
Detrás de la magia de la IA generativa hay modelos de idiomas grandes y técnicas avanzadas de aprendizaje automático. Estos sistemas están capacitados en grandes cantidades de datos, como bibliotecas completas de libros, millones de imágenes, años de música grabada y datos raspados de Internet.
Los desarrolladores de IA, desde gigantes tecnológicos hasta nuevas empresas, son conscientes de que la IA es tan buena como los datos que lo alimenta. Si se alimenta de datos de baja calidad, la IA puede producir resultados sesgados. Es algo con lo que incluso los jugadores más grandes en el campo, como Google, no han sido inmunes.
La IA aprende patrones, relaciones y estructuras dentro de estos datos durante el entrenamiento. Luego, cuando se le solicita, aplica ese conocimiento para generar algo nuevo. Por ejemplo, si le pide a una herramienta Gen AI que escriba un poema sobre el océano, no solo extrae versos preescritos de una base de datos. En cambio, está usando lo que aprendió sobre la poesía, los océanos y la estructura del lenguaje para crear una pieza completamente original.
Es impresionante, pero no es perfecto. A veces los resultados pueden sentirse un poco apagados. Tal vez la IA malinterpreta su solicitud, o se vuelve demasiado creativo de una manera que no esperaba. Puede proporcionar con confianza información completamente falsa, y depende de usted verificarla. Esas peculiaridades, a menudo llamadas alucinaciones, son parte de lo que hace que la IA generativa sea fascinante y frustrante.
Las capacidades generativas de IA están creciendo. Ahora puede comprender múltiples tipos de datos combinando tecnologías como el aprendizaje automático, el procesamiento del lenguaje natural y la visión por computadora. El resultado se llama IA multimodal que puede integrar alguna combinación de texto, imágenes, video y habla dentro de un solo marco, ofreciendo respuestas más contextualmente relevantes y precisas. El modo de voz avanzado de ChatGPT es un ejemplo, al igual que el proyecto Astra de Google.
No hay escasez de herramientas de IA generativas, cada una con su talento único. Estas herramientas han provocado la creatividad, pero también han planteado muchas preguntas además del sesgo y las alucinaciones, como, ¿quién posee los derechos del contenido generado por IA? O qué material es un juego justo o fuera de los límites para que las compañías de IA los usen para capacitar a sus modelos de idiomas; vea, por ejemplo, la demanda del New York Times contra Openai y Microsoft.
Otras preocupaciones, no son asuntos pequeños, implican privacidad, responsabilidad en la IA, los profundos profundos generados por IA y el desplazamiento laboral.
“Escribir, animación, fotografía, ilustración, diseño gráfico: las herramientas de IA ahora pueden manejar todo eso con una facilidad sorprendente. Pero eso no significa que estos roles desaparezcan. Simplemente puede significar que los creativos deberán mejorar y usar estas herramientas para amplificar su propio trabajo”, Fang Liu, profesor de la Universidad de Notre Dame Dame y Coeditor-Chief de las transacciones de ACM en las transacciones de Probabilista, contó el aprendizaje en el poderoso de la máquina probabilística, le dijo a Cetnet.
“También ofrece una forma para las personas que tal vez carecen de la habilidad, como alguien con una visión clara que no puede dibujar, pero que puede describirlo a través de un aviso. Así que no, no creo que interrumpa a la industria creativa. Con suerte, será una co-creación o un aumento, no un reemplazo”.
Otro problema es el impacto en el medio ambiente porque la capacitación de grandes modelos de IA utiliza mucha energía, lo que lleva a grandes huellas de carbono. El rápido ascenso de la Generación AI en los últimos años ha acelerado las preocupaciones sobre los riesgos de la IA en general. Los gobiernos están aumentando las regulaciones de IA para garantizar el desarrollo responsable y ético, especialmente la Ley de IA de la Unión Europea.
Muchas personas han interactuado con los chatbots en el servicio al cliente o han utilizado asistentes virtuales como Siri, Alexa y Google Assistant, que ahora están en la cúspide de convertirse en Gen AI Power Tools. Todo eso, junto con las aplicaciones para ChatGPT, Claude y otras herramientas nuevas, es poner ai en sus manos. Y la reacción pública a la IA generativa se ha mezclado. Muchos usuarios disfrutan de la conveniencia y la creatividad que ofrece, especialmente para cosas como escribir ayuda, creación de imágenes, soporte de tareas y productividad.
Mientras tanto, en la encuesta global de IA 2024 de McKinsey, el 65% de los encuestados dijo que sus organizaciones usan regularmente IA generativa, casi el doble de la cifra reportada solo 10 meses antes. Industrias como la atención médica y las finanzas están utilizando Gen AI para racionalizar las operaciones comerciales y automatizar tareas mundanas.
Como se mencionó, existen preocupaciones obvias sobre la ética, la transparencia, la pérdida de empleos y el potencial del mal uso de los datos personales. Esas son las principales críticas detrás de la resistencia a aceptar la IA generativa.
Y las personas que usan herramientas de IA generativas también encontrarán que los resultados aún no son lo suficientemente buenos para el tiempo. A pesar de los avances tecnológicos, la mayoría de las personas pueden reconocer si el contenido se ha creado utilizando Gen AI, ya sean artículos, imágenes o música.
AI ha secuestrado ciertas frases que siempre he usado, por lo que debo autocorrectar mi escritura a menudo porque puede parecer una IA. Muchos artículos escritos por AI contienen frases como “en la era de”, o todo es un “testimonio de” o un “tapiz de”. La IA carece de la emoción y la experiencia que viene, bueno, ser una vida humana y viviente. Como explicó un artista en Quora, “lo que AI hace no es lo mismo que el arte que evoluciona de un pensamiento en un cerebro humano” y “no se crea a partir de la pasión que se encuentra en un corazón humano”.
La IA generativa no es solo para técnicos o personas creativas. Una vez que obtienes la habilidad de darle indicaciones, tiene el potencial de hacer gran parte del trabajo preliminar por ti en una variedad de tareas diarias.
Digamos que está planeando un viaje. En lugar de desplazarse por páginas de resultados de búsqueda, le pide a un chatbot que planifique su itinerario. En cuestión de segundos, tiene un plan detallado adaptado a sus preferencias. (Ese es el ideal. Por favor, verifique siempre sus recomendaciones).
Un propietario de una pequeña empresa que necesita una campaña de marketing pero que no tiene un equipo de diseño puede usar una IA generativa para crear imágenes llamativas e incluso pedirle que sugiera copia publicitaria.
No ha habido un avance tecnológico que haya causado tal boom desde Internet y, más tarde, el iPhone. A pesar de sus desafíos, la IA generativa es innegablemente transformadora. Está haciendo que la creatividad sea más accesible, ayudando a las empresas a racionalizar los flujos de trabajo e incluso inspirar formas completamente nuevas de pensar y resolver problemas.
Pero quizás lo más emocionante es su potencial, y estamos rascando la superficie de lo que estas herramientas pueden hacer.
ChatGPT es probablemente el ejemplo más popular de IA generativa. Le das un aviso y puede generar texto e imágenes; Código de escritura; Responder preguntas; resumir el texto; borrador de correos electrónicos; y mucho más.
La IA generativa crea contenido nuevo como texto, imágenes o música, mientras que la IA tradicional analiza los datos, reconoce patrones o imágenes y hace predicciones (por ejemplo, en medicina, ciencia y finanzas).
Si busca “CHATGPT” en su navegador, es probable que se tope en sitios web que parecen estar alimentados por OpenAI, pero no lo son. Uno de esos sitios, chat.chatbotapp.ai, ofrece acceso a “GPT-3.5” de forma gratuita y utiliza marca familiar.
Pero aquí está la cosa: no está dirigida por OpenAi. Y, francamente, ¿por qué usar un GPT-3.5 potencialmente falso cuando puedes usar GPT-4O de forma gratuita en el actual ¿Sitio de chatgpt?
In the summer of 2023, Ilya Sutskever, a co-founder and the chief scientist of OpenAI, was meeting with a group of new researchers at the company. By all traditional metrics, Sutskever should have felt invincible: He was the brain behind the large language models that helped build ChatGPT, then the fastest-growing app in history; his company’s valuation had skyrocketed; and OpenAI was the unrivaled leader of the industry believed to power the future of Silicon Valley. But the chief scientist seemed to be at war with himself.
Sutskever had long believed that artificial general intelligence, or AGI, was inevitable—now, as things accelerated in the generative-AI industry, he believed AGI’s arrival was imminent, according to Geoff Hinton, an AI pioneer who was his Ph.D. adviser and mentor, and another person familiar with Sutskever’s thinking. (Many of the sources in this piece requested anonymity in order to speak freely about OpenAI without fear of reprisal.) To people around him, Sutskever seemed consumed by thoughts of this impending civilizational transformation. What would the world look like when a supreme AGI emerged and surpassed humanity? And what responsibility did OpenAI have to ensure an end state of extraordinary prosperity, not extraordinary suffering?
By then, Sutskever, who had previously dedicated most of his time to advancing AI capabilities, had started to focus half of his time on AI safety. He appeared to people around him as both boomer and doomer: more excited and afraid than ever before of what was to come. That day, during the meeting with the new researchers, he laid out a plan.
“Once we all get into the bunker—” he began, according to a researcher who was present.
“I’m sorry,” the researcher interrupted, “the bunker?”
“We’re definitely going to build a bunker before we release AGI,” Sutskever replied. Such a powerful technology would surely become an object of intense desire for governments globally. The core scientists working on the technology would need to be protected. “Of course,” he added, “it’s going to be optional whether you want to get into the bunker.”
Two other sources I spoke with confirmed that Sutskever commonly mentioned such a bunker. “There is a group of people—Ilya being one of them—who believe that building AGI will bring about a rapture,” the researcher told me. “Literally, a rapture.” (Sutskever declined to comment.)
Sutskever’s fears about an all-powerful AI may seem extreme, but they are not altogether uncommon, nor were they particularly out of step with OpenAI’s general posture at the time. In May 2023, the company’s CEO, Sam Altman, co-signed an open letter describing the technology as a potential extinction risk—a narrative that has arguably helped OpenAI center itself and steer regulatory conversations. Yet the concerns about a coming apocalypse would also have to be balanced against OpenAI’s growing business: ChatGPT was a hit, and Altman wanted more.
When OpenAI was founded, the idea was to develop AGI for the benefit of humanity. To that end, the co-founders—who included Altman and Elon Musk—set the organization up as a nonprofit and pledged to share research with other institutions. Democratic participation in the technology’s development was a key principle, they agreed, hence the company’s name. But by the time I started covering the company in 2019, these ideals were eroding. OpenAI’s executives had realized that the path they wanted to take would demand extraordinary amounts of money. Both Musk and Altman tried to take over as CEO. Altman won out. Musk left the organization in early 2018 and took his money with him. To plug the hole, Altman reformulated OpenAI’s legal structure, creating a new “capped-profit” arm within the nonprofit to raise more capital.
Since then, I’ve tracked OpenAI’s evolution through interviews with more than 90 current and former employees, including executives and contractors. The company declined my repeated interview requests and questions over the course of working on my book about it, which this story is adapted from; it did not reply when I reached out one more time before the article was published. (OpenAI also has a corporate partnership with The Atlantic.)
OpenAI’s dueling cultures—the ambition to safely develop AGI, and the desire to grow a massive user base through new product launches—would explode toward the end of 2023. Gravely concerned about the direction Altman was taking the company, Sutskever would approach his fellow board of directors, along with his colleague Mira Murati, then OpenAI’s chief technology officer; the board would subsequently conclude the need to push the CEO out. What happened next—with Altman’s ouster and then reinstatement—rocked the tech industry. Yet since then, OpenAI and Sam Altman have become more central to world affairs. Last week, the company unveiled an “OpenAI for Countries” initiative that would allow OpenAI to play a key role in developing AI infrastructure outside of the United States. And Altman has become an ally to the Trump administration, appearing, for example, at an event with Saudi officials this week and onstage with the president in January to announce a $500 billion AI-computing-infrastructure project.
Altman’s brief ouster—and his ability to return and consolidate power—is now crucial history to understand the company’s position at this pivotal moment for the future of AI development. Details have been missing from previous reporting on this incident, including information that sheds light on Sutskever and Murati’s thinking and the response from the rank and file. Here, they are presented for the first time, according to accounts from more than a dozen people who were either directly involved or close to the people directly involved, as well as their contemporaneous notes, plus screenshots of Slack messages, emails, audio recordings, and other corroborating evidence.
The altruistic OpenAI is gone, if it ever existed. What future is the company building now?
Before ChatGPT, sources told me, Altman seemed generally energized. Now he often appeared exhausted. Propelled into megastardom, he was dealing with intensified scrutiny and an overwhelming travel schedule. Meanwhile, Google, Meta, Anthropic, Perplexity, and many others were all developing their own generative-AI products to compete with OpenAI’s chatbot.
Many of Altman’s closest executives had long observed a particular pattern in his behavior: If two teams disagreed, he often agreed in private with each of their perspectives, which created confusion and bred mistrust among colleagues. Now Altman was also frequently bad-mouthing staffers behind their backs while pushing them to deploy products faster and faster. Team leads mirroring his behavior began to pit staff against one another. Sources told me that Greg Brockman, another of OpenAI’s co-founders and its president, added to the problems when he popped into projects and derailed long-standing plans with last-minute changes.
The environment within OpenAI was changing. Previously, Sutskever had tried to unite workers behind a common cause. Among employees, he had been known as a deep thinker and even something of a mystic, regularly speaking in spiritual terms. He wore shirts with animals on them to the office and painted them as well—a cuddly cat, cuddly alpacas, a cuddly fire-breathing dragon. One of his amateur paintings hung in the office, a trio of flowers blossoming in the shape of OpenAI’s logo, a symbol of what he always urged employees to build: “A plurality of humanity-loving AGIs.”
But by the middle of 2023—around the time he began speaking more regularly about the idea of a bunker—Sutskever was no longer just preoccupied by the possible cataclysmic shifts of AGI and superintelligence, according to sources familiar with his thinking. He was consumed by another anxiety: the erosion of his faith that OpenAI could even keep up its technical advancements to reach AGI, or bear that responsibility with Altman as its leader. Sutskever felt Altman’s pattern of behavior was undermining the two pillars of OpenAI’s mission, the sources said: It was slowing down research progress and eroding any chance at making sound AI-safety decisions.
Meanwhile, Murati was trying to manage the mess. She had always played translator and bridge to Altman. If he had adjustments to the company’s strategic direction, she was the implementer. If a team needed to push back against his decisions, she was their champion. When people grew frustrated with their inability to get a straight answer out of Altman, they sought her help. “She was the one getting stuff done,” a former colleague of hers told me. (Murati declined to comment.)
During the development of GPT‑4, Altman and Brockman’s dynamic had nearly led key people to quit, sources told me. Altman was also seemingly trying to circumvent safety processes for expediency. At one point, sources close to the situation said, he had told Murati that OpenAI’s legal team had cleared the latest model, GPT-4 Turbo, to skip review by the company’s Deployment Safety Board, or DSB—a committee of Microsoft and OpenAI representatives who evaluated whether OpenAI’s most powerful models were ready for release. But when Murati checked in with Jason Kwon, who oversaw the legal team, Kwon had no idea how Altman had gotten that impression.
In the summer, Murati attempted to give Altman detailed feedback on these issues, according to multiple sources. It didn’t work. The CEO iced her out, and it took weeks to thaw the relationship.
By fall, Sutskever and Murati both drew the same conclusion. They separately approached the three board members who were not OpenAI employees—Helen Toner, a director at Georgetown University’s Center for Security and Emerging Technology; the roboticist Tasha McCauley; and one of Quora’s co-founders and its CEO, Adam D’Angelo—and raised concerns about Altman’s leadership. “I don’t think Sam is the guy who should have the finger on the button for AGI,” Sutskever said in one such meeting, according to notes I reviewed. “I don’t feel comfortable about Sam leading us to AGI,” Murati said in another, according to sources familiar with the conversation.
That Sutskever and Murati both felt this way had a huge effect on Toner, McCauley, and D’Angelo. For close to a year, they, too, had been processing their own grave concerns about Altman, according to sources familiar with their thinking. Among their many doubts, the three directors had discovered through a series of chance encounters that he had not been forthcoming with them about a range of issues, from a breach in the DSB’s protocols to the legal structure of OpenAI Startup Fund, a dealmaking vehicle that was meant to be under the company but that instead Altman owned himself.
If two of Altman’s most senior deputies were sounding the alarm on his leadership, the board had a serious problem. Sutskever and Murati were not the first to raise these kinds of issues, either. In total, the three directors had heard similar feedback over the years from at least five other people within one to two levels of Altman, the sources said. By the end of October, Toner, McCauley, and D’Angelo began to meet nearly daily on video calls, agreeing that Sutskever’s and Murati’s feedback about Altman, and Sutskever’s suggestion to fire him, warranted serious deliberation.
As they did so, Sutskever sent them long dossiers of documents and screenshots that he and Murati had gathered in tandem with examples of Altman’s behaviors. The screenshots showed at least two more senior leaders noting Altman’s tendency to skirt around or ignore processes, whether they’d been instituted for AI-safety reasons or to smooth company operations. This included, the directors learned, Altman’s apparent attempt to skip DSB review for GPT-4 Turbo.
By Saturday, November 11, the independent directors had made their decision. As Sutskever suggested, they would remove Altman and install Murati as interim CEO. On November 17, 2023, at about noon Pacific time, Sutskever fired Altman on a Google Meet with the three independent board members. Sutskever then told Brockman on another Google Meet that Brockman would no longer be on the board but would retain his role at the company. A public announcement went out immediately.
For a brief moment, OpenAI’s future was an open question. It might have taken a path away from aggressive commercialization and Altman. But this is not what happened.
After what had seemed like a few hours of calm and stability, including Murati having a productive conversation with Microsoft—at the time OpenAI’s largest financial backer—she had suddenly called the board members with a new problem. Altman and Brockman were telling everyone that Altman’s removal had been a coup by Sutskever, she said.
It hadn’t helped that, during a company all-hands to address employee questions, Sutskever had been completely ineffectual with his communication.
“Was there a specific incident that led to this?” Murati had read aloud from a list of employee questions, according to a recording I obtained of the meeting.
“Many of the questions in the document will be about the details,” Sutskever responded. “What, when, how, who, exactly. I wish I could go into the details. But I can’t.”
“Are we worried about the hostile takeover via coercive influence of the existing board members?” Sutskever read from another employee later.
“Hostile takeover?” Sutskever repeated, a new edge in his voice. “The OpenAI nonprofit board has acted entirely in accordance to its objective. It is not a hostile takeover. Not at all. I disagree with this question.”
Shortly thereafter, the remaining board, including Sutskever, confronted enraged leadership over a video call. Kwon, the chief strategy officer, and Anna Makanju, the vice president of global affairs, were leading the charge in rejecting the board’s characterization of Altman’s behavior as “not consistently candid,” according to sources present at the meeting. They demanded evidence to support the board’s decision, which the members felt they couldn’t provide without outing Murati, according to sources familiar with their thinking.
In rapid succession that day, Brockman quit in protest, followed by three other senior researchers. Through the evening, employees only got angrier, fueled by compounding problems: among them, a lack of clarity from the board about their reasons for firing Altman; a potential loss of a tender offer, which had given some the option to sell what could amount to millions of dollars’ worth of their equity; and a growing fear that the instability at the company could lead to its unraveling, which would squander so much promise and hard work.
Faced with the possibility of OpenAI falling apart, Sutskever’s resolve immediately started to crack. OpenAI was his baby, his life; its dissolution would destroy him. He began to plead with his fellow board members to reconsider their position on Altman.
Meanwhile, Murati’s interim position was being challenged. The conflagration within the company was also spreading to a growing circle of investors. Murati now was unwilling to explicitly throw her weight behind the board’s decision to fire Altman. Though her feedback had helped instigate it, she had not participated herself in the deliberations.
By Monday morning, the board had lost. Murati and Sutskever flipped sides. Altman would come back; there was no other way to save OpenAI.
I was already working on a book about OpenAI at the time, and in the weeks that followed the board crisis, friends, family, and media would ask me dozens of times: What did all this mean, if anything? To me, the drama highlighted one of the most urgent questions of our generation: How do we govern artificial intelligence? With AI on track to rewire a great many other crucial functions in society, that question is really asking: How do we ensure that we’ll make our future better, not worse?
The events of November 2023 illustrated in the clearest terms just how much a power struggle among a tiny handful of Silicon Valley elites is currently shaping the future of this technology. And the scorecard of this centralized approach to AI development is deeply troubling. OpenAI today has become everything that it said it would not be. It has turned into a nonprofit in name only, aggressively commercializing products such as ChatGPT and seeking historic valuations. It has grown ever more secretive, not only cutting off access to its own research but shifting norms across the industry to no longer share meaningful technical details about AI models. In the pursuit of an amorphous vision of progress, its aggressive push on the limits of scale has rewritten the rules for a new era of AI development. Now every tech giant is racing to out-scale one another, spending sums so astronomical that even they have scrambled to redistribute and consolidate their resources. What was once unprecedented has become the norm.
As a result, these AI companies have never been richer. In March, OpenAI raised $40 billion, the largest private tech-funding round on record, and hit a $300 billion valuation. Anthropic is valued at more than $60 billion. Near the end of last year, the six largest tech giants together had seen their market caps increase by more than $8 trillion after ChatGPT. At the same time, more and more doubts have risen about the true economic value of generative AI, including a growing body of studies that have shown that the technology is not translating into productivity gains for most workers, while it’s also eroding their critical thinking.
In a November Bloomberg article reviewing the generative-AI industry, the staff writers Parmy Olson and Carolyn Silverman summarized it succinctly. The data, they wrote, “raises an uncomfortable prospect: that this supposedly revolutionary technology might never deliver on its promise of broad economic transformation, but instead just concentrate more wealth at the top.”
Meanwhile, it’s not just a lack of productivity gains that many in the rest of the world are facing. The exploding human and material costs are settling onto wide swaths of society, especially the most vulnerable, people I met around the world, whether workers and rural residents in the global North or impoverished communities in the global South, all suffering new degrees of precarity. Workers in Kenya earned abysmal wages to filter out violence and hate speech from OpenAI’s technologies, including ChatGPT. Artists are being replaced by the very AI models that were built from their work without their consent or compensation. The journalism industry is atrophying as generative-AI technologies spawn heightened volumes of misinformation. Before our eyes, we’re seeing an ancient story repeat itself: Like empires of old, the new empires of AI are amassing extraordinary riches across space and time at great expense to everyone else.
To quell the rising concerns about generative AI’s present-day performance, Altman has trumpeted the future benefits of AGI ever louder. In a September 2024 blog post, he declared that the “Intelligence Age,” characterized by “massive prosperity,” would soon be upon us. At this point, AGI is largely rhetorical—a fantastical, all-purpose excuse for OpenAI to continue pushing for ever more wealth and power. Under the guise of a civilizing mission, the empire of AI is accelerating its global expansion and entrenching its power.
As for Sutskever and Murati, both parted ways with OpenAI after what employees now call “The Blip,” joining a long string of leaders who have left the organization after clashing with Altman. Like many of the others who failed to reshape OpenAI, the two did what has become the next-most-popular option: They each set up their own shops, to compete for the future of this technology.
This essay has been adapted from Karen Hao’s forthcoming book, Empire of AI.
By Karen Hao
*Illustration by Akshita Chandra / The Atlantic. Sources: Nathan Howard / Bloomberg / Getty; Jack Guez / AFP / Getty; Jon Kopaloff / Getty; Manuel Augusto Moreno / Getty; Yuichiro Chino / Getty.
When you buy a book using a link on this page, we receive a commission. Thank you for supporting The Atlantic.
Remove.bg: La Revolución en la Edición de Imágenes que Debes Conocer
Cómo Comenzar a Utilizar ChatGPT: Una Guía Completa para Principiantes
Cómo Empezar con Popai.pro: Tu Espacio Personal de IA – Guía Completa, Instalación, Versiones y Precios
Startups de IA en EE.UU. que han recaudado más de $100M en 2024
Deepgram: Revolucionando el Reconocimiento de Voz con IA
Perplexity aplicado al Marketing Digital y Estrategias SEO
Suno.com: La Revolución en la Creación Musical con Inteligencia Artificial
Dos periodistas octogenarios deman a ChatGPT por robar su trabajo
Inside the OpenAI-DeepSeek Distillation Saga & Alibaba’s Most Powerful AI Model Qwen2.5-Max
De ChatGPT a mil millones de agentes
