The advent of generative AI based text-to-video is gaining steam, especially by the release of … [+]
getty
In today’s column, I explain the hullabaloo over the advent of text-to-video (T2V) in generative AI apps and large language models (LLM). The upshot is this. There is little doubt that text-to-video is still in its infancy at this time, but, by gosh, keep your eye on the ball because T2V is going to gain significant advances that will ultimately knock the socks off the world. As Dr. Seuss might declare, oh, the things that you can do (hang in there, I’ll cover the possibilities momentarily).
As tangible evidence of what text-to-video can do right now, I’ll include in this discussion an assessment of the newly released OpenAI product Sora Turbo, a cousin of the wildly and widely popular ChatGPT. If you are tempted to try out Sora Turbo, it is initially only being made available to ChatGPT Plus and ChatGPT Pro users, meaning that you must pay-to-play. Sad face.
A notable consideration to keep in mind is that ChatGPT currently garners a reported 300 million weekly active users, and though not all of them are going to have ready access to Sora Turbo, an impressive many millions will. Competing products are likely to find that Sora Turbo becomes the 600-pound gorilla and the elephant in the room. By and large, a massive number of users and a massive amount of media attention is going to shift overnight toward Sora Turbo.
Let’s talk about it.
This analysis of an innovative AI advancement is part of my ongoing Forbes column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here). For my coverage of the top-of-the-line ChatGPT o1 model and its advanced functionality, see the link here and the link here.
I’d like to lay out some foundational aspects so that we can then dive deeply into the text-to-video realm.
Generative AI and LLMs generally began by providing text-to-text (T2T) capabilities. You type in text as a prompt, and the AI responds with text such as an essay, poem, narrative, etc. That’s cool. Another exciting feature consists of text-to-image, whereby you enter a prompt, and the AI generates an image such as a photo-realistic picture, a digital painting, a still cartoon, or other kinds of static imagery. Those two modes of usage are nearly old hat now.
The dream for AI researchers is to allow a person to enter a prompt and then have the AI generate a video. A stripped-down way to do this is to focus solely on the visual video and not include any audio. Gradually, we will see the production of visual video elements that are hand-in-hand accompanied by AI-generated matching audio (some LLMs do this but in quite a limited fashion).
A bonus on top of doing text-to-video is the possibility of taking an image as input and turning that into a video. The image might be by itself as the source content, or the AI might accept both a prompt as text and an accompanying image. Finally, the topmost aim is to allow the use of a separate video as the input source, possibly accompanied by text and images, all of which the generative AI utilizes to produce a suitable video. I refer to that as the all-encompassing full-meal deal.
Notice that I just mentioned that the quest or hope is that the generative AI will produce a suitable video. My emphasis on that point is the nature of suitability.
Suitability is the trickiest part of this grand scheme. Allow me to explain. If someone enters a prompt that tells AI to produce a video about a cat wearing a hat that is sitting in a box and riding on a moving train, I’d like you to take a moment and imagine what that video looks like.
Go ahead, envision away, I’ll wait.
I dare say that if you told someone what the video would precisely look like, their conception of the video is going to be quite adrift from what you had in mind. Sure, you would both undoubtedly include a cat of some kind, a hat of some kind on the head of the cat, a box of some kind with the cat inside, and a moving train of some kind. But all of those might vary dramatically from the other person’s conception. Yours could be photo-realistic while the other person imagined animation. The colors would differ, the sizes and shapes would differ, and the action of the cat and the moving train would differ.
I’m sure you get the picture (aha, a pun).
Suitability or the act of meeting the request posed by the human user is a tough nut to crack. Your first impulse might be that if a person writes a lengthy prompt, that would seemingly narrow things down. It might do so to some extent. On the other hand, the odds are still notably high that there would still be marked differences.
Earlier this year, OpenAI made available on a limited basis their new product Sora. Sora is a generative AI app that does text-to-video. Though it is referred to as text-to-video, it also does allow for the input of images and the input of video.
As an aside, the ultimate aim of AI makers across the board is to have what is known as X-to-X modes for generative AI, meaning that X can be text, images, audio, video, and anything else we come up with. The angle is that the end game consists of taking any type of medium as input and having the AI produce any desired type of medium as the output.
Boom, drop the mic.
No worries, we’ll get there (or, maybe we should be worried, as I’ll bring up toward the end here).
After Sora had its limited availability tryouts, OpenAI made some important changes and has now released the modified and more advanced version, known as Sora Turbo. Clever naming. You might want to go online and watch some posted videos showcasing the use of Sora Turbo. I say that because it is difficult in a written form such as this discussion to convey the look and feel of the prompts and controls you can use, and likewise allow you to see the generated videos. The official Sora portion of the OpenAI website shows some handy examples, plus there are already tons of user-made videos available on social media.
The next aspects that I will cover are the types of features and functionality that we nowadays expect a high-end text-to-video AI app to possess. I bring this up to acquaint you with the ins and outs of AI-based text-to-video capabilities.
In a sense, this is almost as though you are interested in possibly using or buying a car, but you aren’t familiar with the features and functions of automobiles. It can be tough to shop for a car if you are in the dark about what counts.
I will briefly identify some of the keystone elements of text-to-video. In addition, I’ll provide an assigned letter grade for what I perceive of the just-released Sora Turbo capabilities. I want to clarify that my letter grading is based on a first glance. My to-do list consists of spending some dedicated time with Sora Turbo and subsequently doing an in-depth review.
Be on the lookout for that posting.
I already brought up the fact that suitability is the Holy Grail of text-to-video.
Somehow, once the AI parses the input prompt, a video is to be generated that matches what the user has inside their mind. Whoa, we aren’t yet at mind-reading by AI (well, there are efforts underway to create brain-machine interfaces or BMI, see my discussion at the link here).
The AI industry tends to refer to this suitability factor as faithfulness or honesty. The AI is supposed to do a bang-up job and reach a faithful or honest rendering in video format of what the user wants.
I am going to say that all the readily available T2V is still at a grade level of C, including Sora Turbo. Inch by inch, clever techniques are being devised to hone in on what a user wants. This is mainly being done in AI research labs and we will gradually see those capabilities come into the public sphere.
The video that was generated in the early days of text-to-video was very rudimentary. They were mainly low-resolution. The graphics were jerky while in motion. I’m not knocking on those heroic initial efforts. We ought to appreciate the pioneering work else we wouldn’t be where we are today.
Tip of the hat.
My point is that thankfully, we’ve come a long way, baby. If you get a chance to see the Sora Turbo AI-generated videos, the vividness, quality, and resolution are pretty much state-of-the-art for T2V. I’ll give this an A-/B+.
Yes, I am a tough-as-nails grader.
I’m sure that you know that movies consist of individual frames that flow past our eyes so fast that we perceive that there is fluid motion afoot in what we are watching. The conventional text-to-video generation adheres to that same practice. A series of one after one-after-another frames are generated, and when they flow along, you perceive motion.
The rub is this. Suppose that in one frame a cat wearing a hat is at the left side of the view. The next frame is supposed to show the cat moving toward the right side, having moved just a nudge to the right. And so on this goes.
If the AI doesn’t figure out things properly, the next frame might show the cat suddenly at the far right of the view. Oops, you are going to be jostled that the cat somehow miraculously got from the left to the right. It won’t look smooth.
This is generally known as temporal consistency. The AI is to render the contents of the frames so that from one frame to the next, which is based on time as each frame goes past our eyes, there should be appropriate consistency. It is a hard problem, just to let you know. I’ll give Sora Turbo a B and anticipate this will be getting stronger as they continue their advancements.
You are watching an AI-generated video, and it shows a cat wearing a hat. The cat moves toward the right side of the scene. Suddenly, the hat disappears. It vanished. What the heck? This wasn’t part of the text prompt in the sense that the user didn’t say anything about making the hat vanish.
The AI did this.
Parlance for this is that we expect the AI to abide by object permanence and not mess around with things. An object that is shown in one frame should customarily be shown in the next frame, perhaps moved around or partially behind another object, but it ought to normally still be there somewhere. I’ll score Sora Turbo as a B-/C+.
Again, this is a hard problem and is being avidly pursued by everyone in this realm.
This next topic consists of something known as scene physics for text-to-video. It is one of the most beguiling of all capabilities and keeps AI researchers and AI developers up at night. They probably have nightmares, vivid ones.
It goes like this. You are watching an AI-generated video, and a character drops a brittle mug. Here on planet Earth, the mug is supposed to obey the laws of gravity. Down it falls. Kablam, the mug hits the floor in the scene and shatters into a zillion pieces.
That is the essence of scene physics. The kinds of intense calculations needed to figure out which way objects should natively go based on ordinary laws of nature is a big hurdle. In addition, the user might have stated that physics is altered, maybe telling the AI to pretend that the action is occurring on the Moon or Mars. I’ll score Sora Turbo as a B-/C+.
I don’t have the space here to go into the myriad of text-to-video features and functions in modern-day T2V.
To give you a taste of things, here’s a list of many equally important capabilities in T2V products:
One thing you ought to especially be aware of is that T2V right now is usually only generating video that consists of a relatively short length of time. When T2V first came around, the videos were a second or two in length. They were nearly a blink of an eye.
Nowadays, many of the mainstay players can do somewhere around 10 to 20 seconds of video. That’s probably just enough to provide a brief scene, but it certainly doesn’t equate to a full-length movie. You can usually use a sequencing or storyboarding function that allows you to place one generated scene after another. That’s good. The downside currently is that the scenes aren’t likely to line up in a suitable alignment. Scene-to-scene continuity is typically weak and telling.
Overall, across the extensive list above, I’ll say that Sora Turbo is somewhere on an A-/B+ and you’ll find plenty of useful controls and functions to keep you busy and entertained.
Shifting gears, I said at the opening of this discussion that text-to-video is quite a big deal. Let’s do a sobering unpacking of that thought.
Envision that with the use of prompts, just about anyone will eventually be able to produce top-quality videos that match Hollywood movies. This sends shivers down the spine of the entertainment industry. AI is coming at all those movie stars, filmmakers, support crews, and the like. Some in the biz insist that AI will never be able to replicate what human filmmakers can achieve.
Well, it’s debatable.
Furthermore, if you construe that the writer of the prompt is a said-to-be “filmmaker” you could argue that the human still is in the loop. One twist is that there are already efforts toward having generative AI come up with prompts that feed into AI-based text-to-video. Blasphemous.
There is something else of more immediate concern since the likelihood of T2V creating full-length top-notch movies is still a bit further on the horizon. The immediate qualm is that people are going to be able to make deepfakes of an incredibly convincing nature. See my coverage of deepfake-making via the AI tools to date, at the link here and the link here, and what’s likely going to happen with the next wave of AI advances.
Utterly convincing deepfakes will be made upon millions and billions of them. At low or nearly zero cost. They are easily distributed digitally across networks, at a low or negligible cost. They will be extremely hard to differentiate from real-life real-world videos.
At an enormous scale.
Disconcertingly, they will look like they are real-life videos. Consider the ramifications. A person is wanted for a heinous crime and a nationwide hunt is underway. The public is asked to submit videos from ring cams, their smartphones, and anything they have that might help in spotting the individual.
It would be very easy to create a video that seemed to show the person walking down the street in a given city, completely fabricated by using AI-based text-to-video. The video is believed. This might cause people in that area to become panicked. Law enforcement resources might be pulled from other locales to concentrate on where the suspect was last presumably seen.
You get the idea.
In my grab-bag list above of T2V features, I noted that watermarking is a feature that AI makers are including in the generated video, allowing for the potential detection and tracking of deepfakes. It is a cat-and-mouse game where evildoers find ways to defeat the watermarks. Another item listed was the AI maker placing restrictions on what can be included in a generated video, such as not allowing the faces and figures of politicians, celebrities, and so on. Again, there are sneaky ways to try and overcome those restrictions.
If you weren’t thinking about AI ethics and AI laws before now, it is time to put on some serious thinking caps.
To what degree should AI makers have discretion in the controls and limits? Should new AI-related laws be enacted? Will such laws potentially hamper AI advancement and place our country at a disadvantage over others (see my analysis of AI advances as a form of exerting national political power on the world stage, at the link here).
OpenAI acknowledges the disconcerting dilemma and noted this as a significant point in their official webpage about Sora Turbo entitled “Sora Is Here” (posted December 9, 2024): “We’re introducing our video generation technology now to give society time to explore its possibilities and co-develop norms and safeguards that ensure it’s used responsibly as the field advances.”
Yes, we all have a stake in this. Go ahead and get up-to-speed on the latest in text-to-video, and while you are at it, join in spirited and crucial discussions about where this is heading and what we can or ought to do to guide humankind in a suitable direction.
There it is again, the importance of suitability.
Ya sea que se dé cuenta o no, la inteligencia artificial está en todas partes. Se encuentra detrás de los chatbots con los que hablas en línea, las listas de reproducción que transmites y los anuncios personalizados que aparecen en tu desplazamiento. Y ahora está tomando una personalidad más pública. Piense en Meta AI, que ahora está integrado en aplicaciones como Facebook, Messenger y WhatsApp; o Géminis de Google, trabajando en segundo plano en las plataformas de la compañía; o Apple Intelligence, lanzando a través de iPhones ahora.
AI tiene una larga historia, volviendo a una conferencia en Dartmouth en 1956 que primero discutió la inteligencia artificial como una cosa. Los hitos en el camino incluyen Eliza, esencialmente el primer chatbot, desarrollado en 1964 por el informático del MIT Joseph Weizenbaum y, saltando 40 años, cuando la función de autocompleta de Google apareció por primera vez en 2004.
Luego llegó 2022 y el ascenso de Chatgpt a la fama. Los desarrollos generativos de IA y los lanzamientos de productos se han acelerado rápidamente desde entonces, incluidos Google Bard (ahora Gemini), Microsoft Copilot, IBM Watsonx.ai y los modelos de LLAMA de código abierto de Meta.
Desglosemos qué es la IA generativa, cómo difiere de la inteligencia artificial “regular” y si la Generación AI puede estar a la altura de las expectativas.
En esencia, la IA generativa se refiere a sistemas de inteligencia artificial que están diseñados para producir un nuevo contenido basado en patrones y datos que han aprendido. En lugar de solo analizar números o predecir tendencias, estos sistemas generan salidas creativas como texto, música de imágenes, videos y código de software.
Algunas de las herramientas de IA generativas más populares en el mercado incluyen:
El principal entre sus habilidades, ChatGPT puede crear conversaciones o ensayos similares a los humanos basados en algunas indicaciones simples. Dall-E y MidJourney crean obras de arte detalladas a partir de una breve descripción, mientras que Adobe Firefly se centra en la edición y el diseño de imágenes.
No toda la IA es generativa. Si bien Gen AI se enfoca en crear contenido nuevo, la IA tradicional se destaca por analizar datos y hacer predicciones. Esto incluye tecnologías como el reconocimiento de imágenes y el texto predictivo. También se usa para soluciones novedosas en:
La IA que venció a los grandes campeones humanos en el ajedrez y el juego de mesa no fue una IA generativa.
Es posible que estos sistemas no sean tan llamativos como la Generación AI, pero la inteligencia artificial clásica es una gran parte de la tecnología en la que confiamos todos los días.
Detrás de la magia de la IA generativa hay modelos de idiomas grandes y técnicas avanzadas de aprendizaje automático. Estos sistemas están capacitados en grandes cantidades de datos, como bibliotecas completas de libros, millones de imágenes, años de música grabada y datos raspados de Internet.
Los desarrolladores de IA, desde gigantes tecnológicos hasta nuevas empresas, son conscientes de que la IA es tan buena como los datos que lo alimenta. Si se alimenta de datos de baja calidad, la IA puede producir resultados sesgados. Es algo con lo que incluso los jugadores más grandes en el campo, como Google, no han sido inmunes.
La IA aprende patrones, relaciones y estructuras dentro de estos datos durante el entrenamiento. Luego, cuando se le solicita, aplica ese conocimiento para generar algo nuevo. Por ejemplo, si le pide a una herramienta Gen AI que escriba un poema sobre el océano, no solo extrae versos preescritos de una base de datos. En cambio, está usando lo que aprendió sobre la poesía, los océanos y la estructura del lenguaje para crear una pieza completamente original.
Es impresionante, pero no es perfecto. A veces los resultados pueden sentirse un poco apagados. Tal vez la IA malinterpreta su solicitud, o se vuelve demasiado creativo de una manera que no esperaba. Puede proporcionar con confianza información completamente falsa, y depende de usted verificarla. Esas peculiaridades, a menudo llamadas alucinaciones, son parte de lo que hace que la IA generativa sea fascinante y frustrante.
Las capacidades generativas de IA están creciendo. Ahora puede comprender múltiples tipos de datos combinando tecnologías como el aprendizaje automático, el procesamiento del lenguaje natural y la visión por computadora. El resultado se llama IA multimodal que puede integrar alguna combinación de texto, imágenes, video y habla dentro de un solo marco, ofreciendo respuestas más contextualmente relevantes y precisas. El modo de voz avanzado de ChatGPT es un ejemplo, al igual que el proyecto Astra de Google.
No hay escasez de herramientas de IA generativas, cada una con su talento único. Estas herramientas han provocado la creatividad, pero también han planteado muchas preguntas además del sesgo y las alucinaciones, como, ¿quién posee los derechos del contenido generado por IA? O qué material es un juego justo o fuera de los límites para que las compañías de IA los usen para capacitar a sus modelos de idiomas; vea, por ejemplo, la demanda del New York Times contra Openai y Microsoft.
Otras preocupaciones, no son asuntos pequeños, implican privacidad, responsabilidad en la IA, los profundos profundos generados por IA y el desplazamiento laboral.
“Escribir, animación, fotografía, ilustración, diseño gráfico: las herramientas de IA ahora pueden manejar todo eso con una facilidad sorprendente. Pero eso no significa que estos roles desaparezcan. Simplemente puede significar que los creativos deberán mejorar y usar estas herramientas para amplificar su propio trabajo”, Fang Liu, profesor de la Universidad de Notre Dame Dame y Coeditor-Chief de las transacciones de ACM en las transacciones de Probabilista, contó el aprendizaje en el poderoso de la máquina probabilística, le dijo a Cetnet.
“También ofrece una forma para las personas que tal vez carecen de la habilidad, como alguien con una visión clara que no puede dibujar, pero que puede describirlo a través de un aviso. Así que no, no creo que interrumpa a la industria creativa. Con suerte, será una co-creación o un aumento, no un reemplazo”.
Otro problema es el impacto en el medio ambiente porque la capacitación de grandes modelos de IA utiliza mucha energía, lo que lleva a grandes huellas de carbono. El rápido ascenso de la Generación AI en los últimos años ha acelerado las preocupaciones sobre los riesgos de la IA en general. Los gobiernos están aumentando las regulaciones de IA para garantizar el desarrollo responsable y ético, especialmente la Ley de IA de la Unión Europea.
Muchas personas han interactuado con los chatbots en el servicio al cliente o han utilizado asistentes virtuales como Siri, Alexa y Google Assistant, que ahora están en la cúspide de convertirse en Gen AI Power Tools. Todo eso, junto con las aplicaciones para ChatGPT, Claude y otras herramientas nuevas, es poner ai en sus manos. Y la reacción pública a la IA generativa se ha mezclado. Muchos usuarios disfrutan de la conveniencia y la creatividad que ofrece, especialmente para cosas como escribir ayuda, creación de imágenes, soporte de tareas y productividad.
Mientras tanto, en la encuesta global de IA 2024 de McKinsey, el 65% de los encuestados dijo que sus organizaciones usan regularmente IA generativa, casi el doble de la cifra reportada solo 10 meses antes. Industrias como la atención médica y las finanzas están utilizando Gen AI para racionalizar las operaciones comerciales y automatizar tareas mundanas.
Como se mencionó, existen preocupaciones obvias sobre la ética, la transparencia, la pérdida de empleos y el potencial del mal uso de los datos personales. Esas son las principales críticas detrás de la resistencia a aceptar la IA generativa.
Y las personas que usan herramientas de IA generativas también encontrarán que los resultados aún no son lo suficientemente buenos para el tiempo. A pesar de los avances tecnológicos, la mayoría de las personas pueden reconocer si el contenido se ha creado utilizando Gen AI, ya sean artículos, imágenes o música.
AI ha secuestrado ciertas frases que siempre he usado, por lo que debo autocorrectar mi escritura a menudo porque puede parecer una IA. Muchos artículos escritos por AI contienen frases como “en la era de”, o todo es un “testimonio de” o un “tapiz de”. La IA carece de la emoción y la experiencia que viene, bueno, ser una vida humana y viviente. Como explicó un artista en Quora, “lo que AI hace no es lo mismo que el arte que evoluciona de un pensamiento en un cerebro humano” y “no se crea a partir de la pasión que se encuentra en un corazón humano”.
La IA generativa no es solo para técnicos o personas creativas. Una vez que obtienes la habilidad de darle indicaciones, tiene el potencial de hacer gran parte del trabajo preliminar por ti en una variedad de tareas diarias.
Digamos que está planeando un viaje. En lugar de desplazarse por páginas de resultados de búsqueda, le pide a un chatbot que planifique su itinerario. En cuestión de segundos, tiene un plan detallado adaptado a sus preferencias. (Ese es el ideal. Por favor, verifique siempre sus recomendaciones).
Un propietario de una pequeña empresa que necesita una campaña de marketing pero que no tiene un equipo de diseño puede usar una IA generativa para crear imágenes llamativas e incluso pedirle que sugiera copia publicitaria.
No ha habido un avance tecnológico que haya causado tal boom desde Internet y, más tarde, el iPhone. A pesar de sus desafíos, la IA generativa es innegablemente transformadora. Está haciendo que la creatividad sea más accesible, ayudando a las empresas a racionalizar los flujos de trabajo e incluso inspirar formas completamente nuevas de pensar y resolver problemas.
Pero quizás lo más emocionante es su potencial, y estamos rascando la superficie de lo que estas herramientas pueden hacer.
ChatGPT es probablemente el ejemplo más popular de IA generativa. Le das un aviso y puede generar texto e imágenes; Código de escritura; Responder preguntas; resumir el texto; borrador de correos electrónicos; y mucho más.
La IA generativa crea contenido nuevo como texto, imágenes o música, mientras que la IA tradicional analiza los datos, reconoce patrones o imágenes y hace predicciones (por ejemplo, en medicina, ciencia y finanzas).
Si busca “CHATGPT” en su navegador, es probable que se tope en sitios web que parecen estar alimentados por OpenAI, pero no lo son. Uno de esos sitios, chat.chatbotapp.ai, ofrece acceso a “GPT-3.5” de forma gratuita y utiliza marca familiar.
Pero aquí está la cosa: no está dirigida por OpenAi. Y, francamente, ¿por qué usar un GPT-3.5 potencialmente falso cuando puedes usar GPT-4O de forma gratuita en el actual ¿Sitio de chatgpt?
In the summer of 2023, Ilya Sutskever, a co-founder and the chief scientist of OpenAI, was meeting with a group of new researchers at the company. By all traditional metrics, Sutskever should have felt invincible: He was the brain behind the large language models that helped build ChatGPT, then the fastest-growing app in history; his company’s valuation had skyrocketed; and OpenAI was the unrivaled leader of the industry believed to power the future of Silicon Valley. But the chief scientist seemed to be at war with himself.
Sutskever had long believed that artificial general intelligence, or AGI, was inevitable—now, as things accelerated in the generative-AI industry, he believed AGI’s arrival was imminent, according to Geoff Hinton, an AI pioneer who was his Ph.D. adviser and mentor, and another person familiar with Sutskever’s thinking. (Many of the sources in this piece requested anonymity in order to speak freely about OpenAI without fear of reprisal.) To people around him, Sutskever seemed consumed by thoughts of this impending civilizational transformation. What would the world look like when a supreme AGI emerged and surpassed humanity? And what responsibility did OpenAI have to ensure an end state of extraordinary prosperity, not extraordinary suffering?
By then, Sutskever, who had previously dedicated most of his time to advancing AI capabilities, had started to focus half of his time on AI safety. He appeared to people around him as both boomer and doomer: more excited and afraid than ever before of what was to come. That day, during the meeting with the new researchers, he laid out a plan.
“Once we all get into the bunker—” he began, according to a researcher who was present.
“I’m sorry,” the researcher interrupted, “the bunker?”
“We’re definitely going to build a bunker before we release AGI,” Sutskever replied. Such a powerful technology would surely become an object of intense desire for governments globally. The core scientists working on the technology would need to be protected. “Of course,” he added, “it’s going to be optional whether you want to get into the bunker.”
Two other sources I spoke with confirmed that Sutskever commonly mentioned such a bunker. “There is a group of people—Ilya being one of them—who believe that building AGI will bring about a rapture,” the researcher told me. “Literally, a rapture.” (Sutskever declined to comment.)
Sutskever’s fears about an all-powerful AI may seem extreme, but they are not altogether uncommon, nor were they particularly out of step with OpenAI’s general posture at the time. In May 2023, the company’s CEO, Sam Altman, co-signed an open letter describing the technology as a potential extinction risk—a narrative that has arguably helped OpenAI center itself and steer regulatory conversations. Yet the concerns about a coming apocalypse would also have to be balanced against OpenAI’s growing business: ChatGPT was a hit, and Altman wanted more.
When OpenAI was founded, the idea was to develop AGI for the benefit of humanity. To that end, the co-founders—who included Altman and Elon Musk—set the organization up as a nonprofit and pledged to share research with other institutions. Democratic participation in the technology’s development was a key principle, they agreed, hence the company’s name. But by the time I started covering the company in 2019, these ideals were eroding. OpenAI’s executives had realized that the path they wanted to take would demand extraordinary amounts of money. Both Musk and Altman tried to take over as CEO. Altman won out. Musk left the organization in early 2018 and took his money with him. To plug the hole, Altman reformulated OpenAI’s legal structure, creating a new “capped-profit” arm within the nonprofit to raise more capital.
Since then, I’ve tracked OpenAI’s evolution through interviews with more than 90 current and former employees, including executives and contractors. The company declined my repeated interview requests and questions over the course of working on my book about it, which this story is adapted from; it did not reply when I reached out one more time before the article was published. (OpenAI also has a corporate partnership with The Atlantic.)
OpenAI’s dueling cultures—the ambition to safely develop AGI, and the desire to grow a massive user base through new product launches—would explode toward the end of 2023. Gravely concerned about the direction Altman was taking the company, Sutskever would approach his fellow board of directors, along with his colleague Mira Murati, then OpenAI’s chief technology officer; the board would subsequently conclude the need to push the CEO out. What happened next—with Altman’s ouster and then reinstatement—rocked the tech industry. Yet since then, OpenAI and Sam Altman have become more central to world affairs. Last week, the company unveiled an “OpenAI for Countries” initiative that would allow OpenAI to play a key role in developing AI infrastructure outside of the United States. And Altman has become an ally to the Trump administration, appearing, for example, at an event with Saudi officials this week and onstage with the president in January to announce a $500 billion AI-computing-infrastructure project.
Altman’s brief ouster—and his ability to return and consolidate power—is now crucial history to understand the company’s position at this pivotal moment for the future of AI development. Details have been missing from previous reporting on this incident, including information that sheds light on Sutskever and Murati’s thinking and the response from the rank and file. Here, they are presented for the first time, according to accounts from more than a dozen people who were either directly involved or close to the people directly involved, as well as their contemporaneous notes, plus screenshots of Slack messages, emails, audio recordings, and other corroborating evidence.
The altruistic OpenAI is gone, if it ever existed. What future is the company building now?
Before ChatGPT, sources told me, Altman seemed generally energized. Now he often appeared exhausted. Propelled into megastardom, he was dealing with intensified scrutiny and an overwhelming travel schedule. Meanwhile, Google, Meta, Anthropic, Perplexity, and many others were all developing their own generative-AI products to compete with OpenAI’s chatbot.
Many of Altman’s closest executives had long observed a particular pattern in his behavior: If two teams disagreed, he often agreed in private with each of their perspectives, which created confusion and bred mistrust among colleagues. Now Altman was also frequently bad-mouthing staffers behind their backs while pushing them to deploy products faster and faster. Team leads mirroring his behavior began to pit staff against one another. Sources told me that Greg Brockman, another of OpenAI’s co-founders and its president, added to the problems when he popped into projects and derailed long-standing plans with last-minute changes.
The environment within OpenAI was changing. Previously, Sutskever had tried to unite workers behind a common cause. Among employees, he had been known as a deep thinker and even something of a mystic, regularly speaking in spiritual terms. He wore shirts with animals on them to the office and painted them as well—a cuddly cat, cuddly alpacas, a cuddly fire-breathing dragon. One of his amateur paintings hung in the office, a trio of flowers blossoming in the shape of OpenAI’s logo, a symbol of what he always urged employees to build: “A plurality of humanity-loving AGIs.”
But by the middle of 2023—around the time he began speaking more regularly about the idea of a bunker—Sutskever was no longer just preoccupied by the possible cataclysmic shifts of AGI and superintelligence, according to sources familiar with his thinking. He was consumed by another anxiety: the erosion of his faith that OpenAI could even keep up its technical advancements to reach AGI, or bear that responsibility with Altman as its leader. Sutskever felt Altman’s pattern of behavior was undermining the two pillars of OpenAI’s mission, the sources said: It was slowing down research progress and eroding any chance at making sound AI-safety decisions.
Meanwhile, Murati was trying to manage the mess. She had always played translator and bridge to Altman. If he had adjustments to the company’s strategic direction, she was the implementer. If a team needed to push back against his decisions, she was their champion. When people grew frustrated with their inability to get a straight answer out of Altman, they sought her help. “She was the one getting stuff done,” a former colleague of hers told me. (Murati declined to comment.)
During the development of GPT‑4, Altman and Brockman’s dynamic had nearly led key people to quit, sources told me. Altman was also seemingly trying to circumvent safety processes for expediency. At one point, sources close to the situation said, he had told Murati that OpenAI’s legal team had cleared the latest model, GPT-4 Turbo, to skip review by the company’s Deployment Safety Board, or DSB—a committee of Microsoft and OpenAI representatives who evaluated whether OpenAI’s most powerful models were ready for release. But when Murati checked in with Jason Kwon, who oversaw the legal team, Kwon had no idea how Altman had gotten that impression.
In the summer, Murati attempted to give Altman detailed feedback on these issues, according to multiple sources. It didn’t work. The CEO iced her out, and it took weeks to thaw the relationship.
By fall, Sutskever and Murati both drew the same conclusion. They separately approached the three board members who were not OpenAI employees—Helen Toner, a director at Georgetown University’s Center for Security and Emerging Technology; the roboticist Tasha McCauley; and one of Quora’s co-founders and its CEO, Adam D’Angelo—and raised concerns about Altman’s leadership. “I don’t think Sam is the guy who should have the finger on the button for AGI,” Sutskever said in one such meeting, according to notes I reviewed. “I don’t feel comfortable about Sam leading us to AGI,” Murati said in another, according to sources familiar with the conversation.
That Sutskever and Murati both felt this way had a huge effect on Toner, McCauley, and D’Angelo. For close to a year, they, too, had been processing their own grave concerns about Altman, according to sources familiar with their thinking. Among their many doubts, the three directors had discovered through a series of chance encounters that he had not been forthcoming with them about a range of issues, from a breach in the DSB’s protocols to the legal structure of OpenAI Startup Fund, a dealmaking vehicle that was meant to be under the company but that instead Altman owned himself.
If two of Altman’s most senior deputies were sounding the alarm on his leadership, the board had a serious problem. Sutskever and Murati were not the first to raise these kinds of issues, either. In total, the three directors had heard similar feedback over the years from at least five other people within one to two levels of Altman, the sources said. By the end of October, Toner, McCauley, and D’Angelo began to meet nearly daily on video calls, agreeing that Sutskever’s and Murati’s feedback about Altman, and Sutskever’s suggestion to fire him, warranted serious deliberation.
As they did so, Sutskever sent them long dossiers of documents and screenshots that he and Murati had gathered in tandem with examples of Altman’s behaviors. The screenshots showed at least two more senior leaders noting Altman’s tendency to skirt around or ignore processes, whether they’d been instituted for AI-safety reasons or to smooth company operations. This included, the directors learned, Altman’s apparent attempt to skip DSB review for GPT-4 Turbo.
By Saturday, November 11, the independent directors had made their decision. As Sutskever suggested, they would remove Altman and install Murati as interim CEO. On November 17, 2023, at about noon Pacific time, Sutskever fired Altman on a Google Meet with the three independent board members. Sutskever then told Brockman on another Google Meet that Brockman would no longer be on the board but would retain his role at the company. A public announcement went out immediately.
For a brief moment, OpenAI’s future was an open question. It might have taken a path away from aggressive commercialization and Altman. But this is not what happened.
After what had seemed like a few hours of calm and stability, including Murati having a productive conversation with Microsoft—at the time OpenAI’s largest financial backer—she had suddenly called the board members with a new problem. Altman and Brockman were telling everyone that Altman’s removal had been a coup by Sutskever, she said.
It hadn’t helped that, during a company all-hands to address employee questions, Sutskever had been completely ineffectual with his communication.
“Was there a specific incident that led to this?” Murati had read aloud from a list of employee questions, according to a recording I obtained of the meeting.
“Many of the questions in the document will be about the details,” Sutskever responded. “What, when, how, who, exactly. I wish I could go into the details. But I can’t.”
“Are we worried about the hostile takeover via coercive influence of the existing board members?” Sutskever read from another employee later.
“Hostile takeover?” Sutskever repeated, a new edge in his voice. “The OpenAI nonprofit board has acted entirely in accordance to its objective. It is not a hostile takeover. Not at all. I disagree with this question.”
Shortly thereafter, the remaining board, including Sutskever, confronted enraged leadership over a video call. Kwon, the chief strategy officer, and Anna Makanju, the vice president of global affairs, were leading the charge in rejecting the board’s characterization of Altman’s behavior as “not consistently candid,” according to sources present at the meeting. They demanded evidence to support the board’s decision, which the members felt they couldn’t provide without outing Murati, according to sources familiar with their thinking.
In rapid succession that day, Brockman quit in protest, followed by three other senior researchers. Through the evening, employees only got angrier, fueled by compounding problems: among them, a lack of clarity from the board about their reasons for firing Altman; a potential loss of a tender offer, which had given some the option to sell what could amount to millions of dollars’ worth of their equity; and a growing fear that the instability at the company could lead to its unraveling, which would squander so much promise and hard work.
Faced with the possibility of OpenAI falling apart, Sutskever’s resolve immediately started to crack. OpenAI was his baby, his life; its dissolution would destroy him. He began to plead with his fellow board members to reconsider their position on Altman.
Meanwhile, Murati’s interim position was being challenged. The conflagration within the company was also spreading to a growing circle of investors. Murati now was unwilling to explicitly throw her weight behind the board’s decision to fire Altman. Though her feedback had helped instigate it, she had not participated herself in the deliberations.
By Monday morning, the board had lost. Murati and Sutskever flipped sides. Altman would come back; there was no other way to save OpenAI.
I was already working on a book about OpenAI at the time, and in the weeks that followed the board crisis, friends, family, and media would ask me dozens of times: What did all this mean, if anything? To me, the drama highlighted one of the most urgent questions of our generation: How do we govern artificial intelligence? With AI on track to rewire a great many other crucial functions in society, that question is really asking: How do we ensure that we’ll make our future better, not worse?
The events of November 2023 illustrated in the clearest terms just how much a power struggle among a tiny handful of Silicon Valley elites is currently shaping the future of this technology. And the scorecard of this centralized approach to AI development is deeply troubling. OpenAI today has become everything that it said it would not be. It has turned into a nonprofit in name only, aggressively commercializing products such as ChatGPT and seeking historic valuations. It has grown ever more secretive, not only cutting off access to its own research but shifting norms across the industry to no longer share meaningful technical details about AI models. In the pursuit of an amorphous vision of progress, its aggressive push on the limits of scale has rewritten the rules for a new era of AI development. Now every tech giant is racing to out-scale one another, spending sums so astronomical that even they have scrambled to redistribute and consolidate their resources. What was once unprecedented has become the norm.
As a result, these AI companies have never been richer. In March, OpenAI raised $40 billion, the largest private tech-funding round on record, and hit a $300 billion valuation. Anthropic is valued at more than $60 billion. Near the end of last year, the six largest tech giants together had seen their market caps increase by more than $8 trillion after ChatGPT. At the same time, more and more doubts have risen about the true economic value of generative AI, including a growing body of studies that have shown that the technology is not translating into productivity gains for most workers, while it’s also eroding their critical thinking.
In a November Bloomberg article reviewing the generative-AI industry, the staff writers Parmy Olson and Carolyn Silverman summarized it succinctly. The data, they wrote, “raises an uncomfortable prospect: that this supposedly revolutionary technology might never deliver on its promise of broad economic transformation, but instead just concentrate more wealth at the top.”
Meanwhile, it’s not just a lack of productivity gains that many in the rest of the world are facing. The exploding human and material costs are settling onto wide swaths of society, especially the most vulnerable, people I met around the world, whether workers and rural residents in the global North or impoverished communities in the global South, all suffering new degrees of precarity. Workers in Kenya earned abysmal wages to filter out violence and hate speech from OpenAI’s technologies, including ChatGPT. Artists are being replaced by the very AI models that were built from their work without their consent or compensation. The journalism industry is atrophying as generative-AI technologies spawn heightened volumes of misinformation. Before our eyes, we’re seeing an ancient story repeat itself: Like empires of old, the new empires of AI are amassing extraordinary riches across space and time at great expense to everyone else.
To quell the rising concerns about generative AI’s present-day performance, Altman has trumpeted the future benefits of AGI ever louder. In a September 2024 blog post, he declared that the “Intelligence Age,” characterized by “massive prosperity,” would soon be upon us. At this point, AGI is largely rhetorical—a fantastical, all-purpose excuse for OpenAI to continue pushing for ever more wealth and power. Under the guise of a civilizing mission, the empire of AI is accelerating its global expansion and entrenching its power.
As for Sutskever and Murati, both parted ways with OpenAI after what employees now call “The Blip,” joining a long string of leaders who have left the organization after clashing with Altman. Like many of the others who failed to reshape OpenAI, the two did what has become the next-most-popular option: They each set up their own shops, to compete for the future of this technology.
This essay has been adapted from Karen Hao’s forthcoming book, Empire of AI.
By Karen Hao
*Illustration by Akshita Chandra / The Atlantic. Sources: Nathan Howard / Bloomberg / Getty; Jack Guez / AFP / Getty; Jon Kopaloff / Getty; Manuel Augusto Moreno / Getty; Yuichiro Chino / Getty.
When you buy a book using a link on this page, we receive a commission. Thank you for supporting The Atlantic.
Remove.bg: La Revolución en la Edición de Imágenes que Debes Conocer
Cómo Comenzar a Utilizar ChatGPT: Una Guía Completa para Principiantes
Startups de IA en EE.UU. que han recaudado más de $100M en 2024
Cómo Empezar con Popai.pro: Tu Espacio Personal de IA – Guía Completa, Instalación, Versiones y Precios
Deepgram: Revolucionando el Reconocimiento de Voz con IA
Perplexity aplicado al Marketing Digital y Estrategias SEO
Suno.com: La Revolución en la Creación Musical con Inteligencia Artificial
Dos periodistas octogenarios deman a ChatGPT por robar su trabajo
Inside the OpenAI-DeepSeek Distillation Saga & Alibaba’s Most Powerful AI Model Qwen2.5-Max
De ChatGPT a mil millones de agentes
