Hello and welcome to Sync #494!

As part of the court case between Elon Musk and Sam Altman, several emails from the early years of OpenAI have been made public. We will take a look into those emails, and what they reveal about the early days of OpenAI and the growing distrust among its founders.

In other news, Anthropic and xAI raised billions of dollars while a Chinese lab released its reasoning model to challenge OpenAI o1, and Gemini gets memory. Elsewhere in AI, DeepMind releases an AI model to correct errors in quantum computing, a Swiss church installs AI Jesus and how can a small AI control much more capable AI and make sure it does not misbehave.

Over in robotics, we have a fluffy robot from Japan aimed at replacing living pets, how easy it is to jailbreak LLM-powered robots and a South Korean robot dog completes a full marathon.

We’ll wrap up this week’s issue of Sync with a behind-the-scenes tour of the workshop where Wing designs and builds its delivery drones.

Enjoy!

The last two years were quite eventful for OpenAI. Thanks to the massive success of ChatGPT, OpenAI went from a relatively small company to one of the biggest startups in the world, attracting worldwide attention and billions of dollars in funding, propelling its valuation to $157 billion and triggering the AI revolution we are in today.

However, such massive growth never comes easily. Every organisation experiencing significant growth must evolve and transform itself into a new organisation that can deal with new challenges. In the case of OpenAI, some of those growing pains were made publically visible.

Probably the best known of these growing pains occurred a year ago when a group of OpenAI board members briefly removed Sam Altman from his role as CEO. Altman eventually returned to OpenAI, and from then on, we have seen a steady stream of key people leaving the company. Most notable of them were Ilya Sutskever, one of the founders of OpenAI and its long-time chief scientist, and Mira Murati, who served as CTO and briefly as CEO.

As part of the court case between Elon Musk and Sam Altman, several emails from the early years of OpenAI have been made public, revealing the tension within OpenAI has been there since the very beginning of the company. These emails cover the period from 2015, when the idea of an “open” AI lab was first proposed, to 2019, when OpenAI transitioned from a non-profit to a for-profit company. They also complement an earlier batch of communications between Elon Musk, Sam Altman, Greg Brockman, and Ilya Sutskever, released earlier this year by OpenAI.

The emails can be found on the Musk v. Altman court case page. There is also a compilation of all emails on LessWrong which is much easier to read. While reading those emails, it is worth keeping in mind that Musk’s legal team released them so they will be skewed towards portraying him as the one who was betrayed by OpenAI when the company abandoned its original vision of being a non-profit AI research lab.

Before we dive into the emails and what they tell us about OpenAI, let’s remind ourselves what the world looked like when OpenAI was founded.

It is 2015—the height of the deep learning revolution. A year earlier, Google had acquired DeepMind, a London-based AI research company making breakthrough after breakthrough in AI and advancing steadily in deep learning and reinforcement learning research. With the acquisition of DeepMind, Google was poised to lead the charge in AI research. It had the best talent working in its labs, backed by Google’s vast resources—be it computing power, data, or finances. If anyone were to create AGI, there was a big chance it would happen at Google.

That vision of the future, in which AGI has been created and owned by Google, was something Sam Altman did not want to happen. As he wrote in an email to Elon Musk in May 2015:

If it’s going to happen anyway, it seems like it would be good for someone other than Google to do it first.

With the information we have so far, that email is the first time an idea for a non-profit AI lab bringing the best minds in the industry to create advanced AI to benefit all of humanity was proposed. Initially, that company was to be attached to YCombinator and was provisionally named YC AI before eventually being renamed to OpenAI.

Later emails describe how the new company was planning to attract top talent in AI. That’s where Altman mentions that, apparently, DeepMind was trying to “kill” OpenAI by offering massive counteroffers to those who joined the new company.

One thing that comes out from reading those emails is the tension between the founders of OpenAI—Elon Musk, Sam Altman, Ilya Sutskever and Greg Brockman. The best example of those tensions is an email titled “Honest Thoughts,” written by Ilya Sutskever and sent to both Elon Musk and Sam Altman in September 2017. Sutskever’s thoughts were indeed honest. In the email, he openly questioned the motives of both Musk and Altman and their intentions for OpenAI.

Addressing Musk, Sutskever expressed concerns about the possibility of Musk taking control of OpenAI and transforming the AI lab into one of his companies. Sutskever noted that such a scenario would go against the very principles upon which OpenAI was founded.

The goal of OpenAI is to make the future good and to avoid an AGI dictatorship. You are concerned that Demis could create an AGI dictatorship. So do we. So it is a bad idea to create a structure where you could become a dictator if you chose to, especially given that we can create some other structure that avoids this possibility.

Elon did not take that email very well, saying “This is the final straw,” and threatening to leave the company and withdraw his funding, which he officially did six months later.

In the same message, Sutskever also raises concerns about Altman and openly asks what he wants from OpenAI:

We haven’t been able to fully trust your judgements throughout this process, because we don’t understand your cost function.

Following this, Sutskever questions further:

We don’t understand why the CEO title is so important to you. Your stated reasons have changed, and it’s hard to really understand what’s driving it.

Is AGI truly your primary motivation? How does it connect to your political goals? How has your thought process changed over time?

We can see in these questions the seeds of mistrust that will grow over time and eventually culminate in Sutskever leading a group of OpenAI board members to remove Sam Altman from the company in November 2023 over Altman not being “consistently candid.”

Another example of rifts forming between OpenAI founders can be found during the discussions about the future of the company as a non-profit, where Sam Altman is reported to have “lost a lot of trust with Greg and Ilya through this process.”

Another of the topics raised in those emails was the question of funding. Running a cutting-edge AI research lab and hiring top AI talent is expensive. The situation was made worse by the fact that OpenAI was going against Google which could easily outspent OpenAI. In 2017, OpenAI spent $7.9 million—equivalent to a quarter of its functional expenses—on cloud computing alone. By contrast, DeepMind’s total expenses in the same year were $442 million.

The emails reveal discussions about ways to raise additional funds to sustain OpenAI. One idea proposed was an ICO, which emerged in 2018 during one of crypto’s many bubbles. However, the idea was quickly abandoned.

The biggest problem in securing more funding was the non-profit nature of OpenAI. The option that OpenAI eventually took was to become in 2019 a “capped” for-profit, with the profit being capped at 100 times any investment. But there was another option on the table—to bring OpenAI under Tesla.

This option was suggested by Andrej Karpathy in an email titled “Top AI Institutions Today,” dated January 2018. At the time, Karpathy was no longer working at OpenAI and was serving as Tesla’s Director of Artificial Intelligence, reporting directly to Elon Musk. In the email, Karpathy provided an analysis of the AI industry in 2018 and correctly highlighted the massive cost of developing world-class AI. He criticised a for-profit approach, arguing that it would require creating a product, which would divert the focus from AI research. In Karpathy’s view, the only viable path for OpenAI to succeed was to become part of Tesla, with Tesla serving as OpenAI’s “cash cow.”

Elon then forwarded Karpathy’s analysis to Ilya Sutskever and Greg Brockman, adding that:

in my and Andrej’s opinion, Tesla is the only path that could even hope to hold a candle to Google. Even then, the probability of being a counterweight to Google is small. It just isn’t zero

However, Altman, Sutskever and Brockman did not want to become Tesla’s equivalent of DeepMind. Sutskever pointed out in that “Honest Thoughts” email that OpenAI being part of Tesla would conflict with the company’s founding principles. If that were to happen, OpenAI would be answerable to Tesla’s shareholders and obligated to maximize their investments.

A similar distrust of Tesla’s involvement in OpenAI is evident when OpenAI considered the idea of acquiring Cerebras, a startup designing AI chips. In the same “Honest Thoughts” email, Sutskever asserted that the acquisition would most likely be carried out through Tesla and questioned Tesla’s potential involvement, portraying it as another example of Elon Musk attempting to exert greater control over OpenAI.

The released emails end in March 2019, on the same day OpenAI announced the transition from a non-profit to a capped for-profit company. The final communication is between Elon Musk and Sam Altman, with Musk requesting that it be made clear he has no financial interest in OpenAI’s for-profit arm.

A few months later, OpenAI partnered with Microsoft, which invested $1 billion into the AI lab and the long-term partnership between these companies began. In the years that followed, OpenAI shifted its focus to researching transformer models and developing the GPT family of large language models. This journey ultimately culminated in the launch of a small project called ChatGPT—and the rest is history.

OpenAI was founded with the mission of building AGI to benefit all of humanity. However, behind this grand mission lies a story of egos, personalities and different visions of what this mission is really about. After reading those emails, a picture of broken foundations emerges, and the story of clashing visions within the company is still unfolding.

If you enjoy this post, please click the button or share it.

Share

Do you like my work? Consider becoming a paying subscriber to support it

Become a paid subscriber

For those who prefer to make a one-off donation, you can ‘buy me a coffee’ via Ko-fi. Every coffee bought is a generous support towards the work put into this newsletter.

Your support, in any form, is deeply appreciated and goes a long way in keeping this newsletter alive and thriving.

A $12,000 Surgery to Change Eye Color Is Surging in Popularity
Keratopigmentation (corneal tattooing) is a cosmetic surgery that permanently changes eye colour by embedding dye into the cornea using a laser. The procedure, which can cost $12,000, has surged in popularity, with one US surgeon reporting growth from 15 to 400 patients annually since 2019. Those who have undergone the procedure say they look better and feel more confident. However, the American Academy of Ophthalmology warns of serious risks, including vision loss, and advises against the procedure for cosmetic purposes.

Neuralink gets approval to start human trials in Canada
Neuralink has received Health Canada’s approval to conduct its first human clinical trials outside the United States. The trials, to be held at Toronto Western Hospital, will test the company’s brain implant technology, which allows people with quadriplegia to control devices using their thoughts. The trials will assess the implant’s safety and functionality, building on earlier patient experiences and addressing issues such as thread retraction. Neuralink is currently recruiting participants with limited hand mobility due to spinal cord injuries or ALS.

Six startups connecting brains with machines

US government commission pushes Manhattan Project-style AI initiative
The US-China Economic and Security Review Commission (USCC) proposed a Manhattan Project-style initiative to develop AGI. The commission emphasized the critical importance of being first in AGI development to maintain a global power balance, particularly in the context of competition with China, with energy infrastructure and streamlined permitting for data centres being cited as key bottlenecks for accelerating AI development. The initiative would involve public-private partnerships, though no specific investment strategies were detailed. OpenAI also called for increased government funding for AI, aligning with the USCC’s focus on public-private collaboration to accelerate AI advancements.

Amazon doubles down on AI startup Anthropic with $4bn investment
Anthropic has announced that Amazon, its long-time backer, is investing another $4 billion into the company, bringing Amazon’s total investments to $8 billion. Amazon will maintain its position as a minority investor and Amazon Web Service (AWS) will become Anthropic’s official cloud provider. Anthropic also said it was working with AWS’s Annapurna Labs on the development of future generations of Amazon’s Trainium chips and plans to train its foundational models on the hardware.

A Chinese lab has released a ‘reasoning’ AI model to rival OpenAI’s o1

Chinese AI lab DeepSeek has released a preview of DeepSeek-R1, designed to rival OpenAI’s o1. Like o1, the model can “think” for tens of seconds to reason through complex questions and tasks. And like o1, it also struggles with certain logic problems, such as tic-tac-toe. Additionally, it can be easily jailbroken, allowing users to bypass its safeguards. DeepSeek claims that DeepSeek-R1 is competitive with OpenAI’s o1-preview on AI benchmarks like AIME (model evaluation) and MATH (word problems).

Elon Musk’s xAI Startup Is Valued at $50 Billion in New Funding Round
xAI has informed investors that it raised $5 billion in a funding round, valuing the company at $50 billion—more than twice its valuation several months ago. In total, xAI has raised $11 billion this year. The company plans to use the new funds in part to finance the purchase of 100,000 additional Nvidia chips to train AI models at its Memphis data centre—the most powerful AI cluster in the world, which Elon Musk has said has plans to double in size.

Google’s Gemini chatbot now has memory
A new “memory” feature has begun rolling out to certain Gemini users. With it, Gemini can now remember facts about the user, their work or their preferences. A similar feature was previously added to ChatGPT which adds more context to the conversation. Gemini memories aren’t used for model training. “Your saved information is never shared or used to train the model,” a Google spokesperson told TechCrunch.

An Honest Review of Apple Intelligence… So Far (17:48)

In this video, Marques Brownlee reviews every single Apple Intelligence feature that’s out so far—Writing Tools, notification summaries and priority notifications, Genmoji, Image Playground, photo cleanup tool in the Photo app, recording summaries, Visual Intelligence and ChatGPT integration. The results are mixed at most.

Microsoft Signs AI-Learning Deal With News Corp.’s HarperCollins
Bloomberg reports that Microsoft has signed a deal with News Corp.’s HarperCollins to use its nonfiction books to train new AI models. This agreement is another example of AI companies signing content partnerships with publishers. OpenAI has similar licensing deals with News Corp., Axel Springer SE, The Atlantic, Vox Media Inc., Dotdash Meredith Inc., Hearst Communications Inc., and Time magazine. Meanwhile, Microsoft has worked on AI initiatives with Reuters, Hearst, and Axel Springer.

Nvidia earnings: AI chip leader shows no signs of stopping mammoth growth
It seems Nvidia just can’t stop growing. The chip maker reported Q3 revenue of $35.08 billion, exceeding expectations of $33.15 billion. Revenue surged 94% year-over-year, with profits more than doubling. The stock is up nearly 200% in 2023 and over 1,100% in the last two years, hitting record highs and propelling Nvidia to become the world’s most valuable company. Nvidia projected a 70% revenue increase for the next quarter, driven by strong demand for Nvidia’s latest Blackwell GPU chips.

AlphaQubit tackles one of quantum computing’s biggest challenges
AlphaQubit is a new model from Google DeepMind trained to identify errors and improve the reliability of quantum computations. According to the blog post, AlphaQubit achieved greater precision than existing error correction systems. However, AlphaQubit is too slow to correct errors in real-time for superconducting quantum processors, which operate at extremely high speeds. Future improvements will focus on speed, scalability, and data-efficient training for systems with millions of qubits.

Ben Affleck Says Movies ‘Will Be One of the Last Things Replaced by AI,’ and Even That’s Unlikely to Happen: ‘AI Is a Craftsman at Best’
Ben Affleck is confident that “movies will be one of the last things, if everything gets replaced, to be replaced by AI.” Affleck explained in an interview that AI is adept at imitation but lacks originality and the ability to create something truly new, akin to how a craftsman works by replicating techniques rather than innovating. However, he sees the benefits of using AI in filmmaking, suggesting AI could lower costs, reduce barriers to entry, and enable more voices to participate in filmmaking by handling logistical and less creative aspects.

Using Dangerous AI, But Safely? (30:37)

Robert Miles, one of the top voices in AI safety, explains a paper asking how can we make sure that a powerful AI model is not trying to be malicious. In this case, the researchers proposed and evaluated various safety protocols within a controlled scenario where an untrusted model (GPT-4) generates code, a trusted but less capable model (GPT-3.5) monitors it, and limited high-quality human labour audits suspicious outputs. The goal is to prevent “backdoors” while maintaining the usefulness of the model’s outputs. It is an interesting video to watch as Miles explains different back-and-forth techniques, resulting in a safety protocol that offers a practical path forward. However, as always, more research is needed to tackle real-world tasks, improve oversight, and address the growing gap between trusted and untrusted models.

Deus in machina: Swiss church installs AI-powered Jesus
Peter’s Chapel in Lucerne, Switzerland, launched Deus in Machina, an AI-powered Jesus avatar capable of dialoguing in 100 languages. The AI was first trained on theological texts and designed to respond to questions in real-time and then installed in the confessional booth for people to interact with it. The project aimed to provide a space for religious conversations and gauge public interest in AI-based spiritual tools. During the two-month trial, over 1,000 people, including tourists and non-Christians, interacted with the avatar. Feedback from 230 users showed two-thirds found the experience spiritual and positive, though others criticized it as superficial or repetitive. The project gathered negative feedback from some church members who criticized the use of the confessional booth and the use of AI in a religious context, raising concerns about the risk of the AI providing inappropriate, illegal, or theologically conflicting responses.

AI-Driven Drug Shows Promising Phase IIa Results in Treating Fatal Lung Disease
ISM001-055, a drug for treating idiopathic pulmonary fibrosis, which was developed with the help of generative AI, is showing promising results in a Phase IIa trial. Created by Insilico Medicine, the drug demonstrated improvements in lung function and quality of life over 12 weeks, offering hope for slowing or potentially reversing disease progression. Insilico Medicine plans to advance to a Phase III trial to further validate the drug’s efficacy and safety. Full trial data will be presented at medical conferences and submitted for peer-reviewed publication.

If you’re enjoying the insights and perspectives shared in the Humanity Redefined newsletter, why not spread the word?

Refer a friend

Can a fluffy robot really replace a cat or dog? My weird, emotional week with an AI pet

I first mentioned Moflin, Casio’s pet robot, in Issue #490. It is an interesting project aiming to create a robot companion that can form an emotional bond with its owner. This article recounts the author’s experience with Moflin, beginning with curiosity and slight self-consciousness, which evolved into subtle attachment as they found its sounds, movements, and interactions comforting. Moments of bonding, such as stroking Moflin or having it rest on their chest, highlighted its ability to provide companionship, although it couldn’t fully replicate the connection of a living pet. Moflin reflects Japan’s growing interest in robotic companions, particularly as solutions for an ageing population, and represents a modern take on the global trend of robotic pets like Sony’s Aibo and Paro the robot seal.

It’s Surprisingly Easy to Jailbreak LLM-Driven Robots
A group of scientists found an automated way to hack into LLM-driven robots with 100% success. By bypassing security measures put into those robots, researchers were able to manipulate self-driving systems into colliding with pedestrians and robot dogs into hunting for harmful places to detonate bombs. Their findings raise concerns about the risks posed by LLM-operated robots in real-world settings and highlight the lack of contextual and consequential awareness in LLMs, emphasizing the need for human oversight in sensitive or safety-critical environments.

Robot runs marathon in South Korea, apparently the first time this has happened
South Korean robot Raibo2 has become the first four-legged robot to complete a full marathon. Built by the Korea Advanced Institute of Science and Technology, the robot completed the 42 km (26.2 miles) run in 4 hours, 19 minutes, and 52 seconds on a single battery charge. For comparison, the human winner finished the marathon in 2 hours, 36 minutes, and 32 seconds.

Scientists identify tomato genes to tweak for sweeter fruit
Researchers in China identified two genes that act as “sugar brakes,” limiting sugar production in domesticated tomatoes during ripening. The resulting tomatoes had up to 30% higher sugar content without sacrificing fruit size or yield, balancing the needs of consumers (who prefer sweet tomatoes) with those of producers and farmers (who want high yield and bigger fruits). The gene-edited tomatoes could be available in supermarkets within 3–5 years. Similar gene-edited tomatoes are already on the market in Japan.

Adam Savage Explores Wing’s Drone Engineering Workshop (26:22)

In this video, Adam Savage visits the workshop and laboratory where Wing, Alphabet’s drone delivery company, designs, builds and tests its delivery drones. Adam learns how these drones were developed, from early prototypes to the machines now delivering packages to real customers. It is fascinating to learn what kind of engineering and problem-solving went into creating a viable delivery drone. Plus it is always a pleasure to see Adam nerding out about exceptional engineering.

Thanks for reading. If you enjoyed this post, please click the button or share it.

Share

Humanity Redefined sheds light on the bleeding edge of technology and how advancements in AI, robotics, and biotech can usher in abundance, expand humanity’s horizons, and redefine what it means to be human.

A big thank you to my paid subscribers, to my Patrons: whmr, Florian, dux, Eric, Preppikoma and Andrew, and to everyone who supports my work on Ko-Fi. Thank you for the support!

My DMs are open to all subscribers. Feel free to drop me a message, share feedback, or just say “hi!”