OpenAI Newly Released AI Product ‘Swarm’ Swiftly Brings Agentic AI Into The Real World

In today’s column, I examine the newly announced OpenAI product called Swarm and explain how this significant unveiling brings the emerging realm of agentic AI into tangible reality.

There is increasing momentum regarding agentic AI as the future next-stretch for the advent of advances in generative AI and large language models or LLMs. Anyone interested in where AI is going ought to be up-to-speed about Swarm since it comes from OpenAI, the 600-pound gorilla or big whale when it comes to advances in generative AI.

Let’s talk about it.

This analysis of an innovative proposition is part of my ongoing Forbes.com column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here).

Agentic AI Fundamentals

Before I do the unpacking of Swarm, I want to make sure we are all on the same page about agentic AI. I’ll provide the keystones of interest. For my detailed coverage of agentic AI, see the link here.

Here’s the deal.

Imagine that you are using generative AI to plan a vacation trip. You would customarily log into your generative AI account such as making use of the widely popular ChatGPT by OpenAI. The planning of your trip would be easy-peasy due to the natural language fluency of ChatGPT. All you need to do is describe where you want to go, and then seamlessly engage in a focused dialogue about the pluses and minuses of places to stay and the transportation options available.

When it comes to booking your trip, the odds are you would have to exit generative AI and start accessing the websites of the hotels, amusement parks, airlines, and other locales to buy your tickets. Few of the major generative AI available today will take that next step on your behalf. It is up to you to perform those tasks.

This is where agents and agentic AI come into play.

In earlier days, you would undoubtedly phone a travel agent to make your bookings. Though there are still human travel agents, another avenue would be to use an AI-based agent that is based on generative AI. The AI has the interactivity that you expect with generative AI. It also has been preloaded with a series of routines or sets of tasks that underpin the efforts of a travel agent. Using everyday natural language, you interact with the agentic AI which works with you on your planning and can proceed to deal with the nitty-gritty of booking your travel plans.

As a use case, envision that there is an overall AI agent that will aid your travel planning and booking. This agentic AI might make use of other AI agents to get the full job done for you. For example, there might be an AI agent booking hotels and doing nothing other than that specific task. Another AI agent books flights. And so on.

The overarching AI travel agent app would invoke or handoff phases of the travel booking activity to the respective AI agents. Those AI agents would perform their particular tasks and then go back to the overarching AI travel agent to indicate how things went.

You could say that the AI travel agent app is orchestrating the overall planning and booking process. This is done via a network of associated AI agents that undertake specialized tasks. The AI agents communicate with each other by passing data back and forth. For example, you might have given your name and credit card info to the AI travel agent app and it passes that along to the AI agent booking the hotel and the AI agent booking your flights.

In a sense, the AI agents are collaborating with each other. I somewhat hesitate to use the word “collaborate” because that might imply a semblance of sentience and overly anthropomorphize AI. Let’s just agree that the AI agents are computationally interacting with each other during the processing of these tasks. We will be a bit generous and suggest they are being collaborative.

Those Agentic AI Advantages

The beauty of this arrangement is that if the AI agents are all based on generative AI, the setup can make use of natural language to bring all the agents together and engage them in working with you interactively. A normal computer program that isn’t based on natural language capabilities would either not interact in a natural language manner, or the collaboration between the various routines or separate apps would have to be programmatically devised.

These AI agents can also make use of tools during their processing. The AI travel agent might have a backend database that keeps track of your various trips. To access the database, the AI travel agent invokes a tool that was built to record data in the database. By using such tools, each AI agent can leverage other available programs that aren’t necessarily natural language based.

I have now introduced you to some of the key terminology associated with agentic AI, consisting of these six primary considerations:

• (1) Orchestration. A generative AI agent will at times orchestrate the use of other AI agents and conduct them toward fulfilling a particular purpose or goal.
• (2) Network of AI agents. Various AI agents are often considered part of a virtual network that allows them to readily access each other.
• (3) Communicate with each other. AI agents are typically set up to communicate with each other by passing data back and forth and performing handoffs with each other to get things done.
• (4) Collaborate with each other. AI agents work in concert or collaborate, though not quite as robustly as humans would, so we’ll loosely say the AI kind of collaborates computationally, including doing handoffs and passing data to each other.
• (5) Autonomously perform tasks. AI agents are said to be at times autonomous in that a human does not necessarily need to be in the loop when the various tasks are being performed by the AI.
• (6) Expressed in natural language. The beauty of AI agents that are devised or based on the use of natural language is that rather than having to laboriously write program code to get them to do things, the use of natural language can be leveraged instead.

Shifting Into The OpenAI Swarm

OpenAI recently announced and made available access to their new product known as Swarm.

I will be quoting from the OpenAI blog about Swarm as posted on October 9, 2024. For those of you interested in actively trying out Swarm, right now it is considered experimental, and you’ll need to use the code that OpenAI has made available on GitHub. If you have sufficient Python coding skills and know how to make use of the generative AI APIs or application programming interface capabilities, you should be able to quickly try out the new product.

This is a one-liner by OpenAI that describes what Swarm is:

• “An educational framework exploring ergonomic, lightweight multi-agent orchestration.”

Swarm is essentially an experimental and educational setup to get agentic AI underway by OpenAI and provides AI developers with a means of trying out agentic AI capabilities. I suppose that the name Swarm refers to the idea that you can have a whole bunch of AI agents working together. In addition, if you think of swarms such as a swarm of bees, swarms often have some overall purpose, such as bees defending against a perceived invader.

The OpenAI blog description quoted above says that the AI agents are lightweight. This suggests that the AI agents are somewhat narrowly scoped and not heavy-duty in terms of any particular agent doing a huge amount of work entirely on its own. That is also where the multi-agent aspects come to the fore. You are presumably going to use lots of said-to-be lightweight AI agents and orchestrate them together to achieve a noted end goal.

An Example Of Agentic AI In Action

The GitHub site and the blog about Swarm showcase some examples of how things work. I have opted to make up my own example and loosely based it on the official ones they posted. I am going to leave out the Python coding to make this example easier to comprehend. By and large, the example generally exemplifies the core essence involved.

My scenario is this.

A company I’ll name as the Widget Corporation wants to develop an automated Customer Support Agent using generative AI. This will be made available to existing customers. A customer will interact directly with the AI agent. The AI agent will find out what the customer’s concerns are. Based on those concerns, the AI agent will attempt to provide a potential resolution. If a resolution is not feasible, the customer will be able to return the item that they bought and get a refund.

I’d dare say this is a pretty common task and usually involves a series of subtasks.

The usual approach for a software developer would be to code this from scratch. It could take gobs of hours to write the code, test it, and field it. Instead, we will use agentic AI and indicate the primary agent, a Customer Support Agent, via the use of natural language.

To illustrate the notion of communication and collaboration, I will define two agents, a Customer Support Agent (considered an AI agentic “Routine” and my primary agent) and a second agent that is Refunds And Returns Agent (considered another AI agentic “Routine” and used by the primary agent). They will do handoffs and make use of tools.

Here is my definition of the Customer Support Agent.

• AI agent routine with tool use and a handoff: Customer Support Agent

“You are a customer support agent for the Widget Corporation.”

“Follow this standard routine:

“(1) When a customer contacts you, make sure to ask sufficient questions to grasp what their customer support issue consists of.”

“(2) Access the Widget Corp internal customer support database WidgetSys to see if any similar issues have ever been logged.”

“(3) Try to come up with a solution for the customer that will resolve their support issue.”

“(4) Provide the proposed solution to the customer and get their feedback.”

“(5) If the customer wants to do a product return and get a refund then invoke the Returns And Refunds Agent and provide relevant details about the customer.”

End of definition

I’d like you to notice that the definition is written in natural language.

If you fed that same text into generative AI such as ChatGPT as a prompt, the AI would generally be able to proceed.

Give that a reflective moment of thought. Imagine the vast amount of arduous coding or programming you would have to write to do the same thing. All we had to do here was express what we wanted via the use of everyday natural language.

Boom, drop the mic.

Inside the natural language description in Step #2, I refer to a tool, the WidgetSys tool. This is a program that the Widget Corporation has developed to access its internal customer service records database.

In Step #5, I mention another AI agent, known as the Returns And Refunds Agent. This is a handoff activity that will occur when Step #5 is performed. In addition, I indicated that relevant customer data should be passed over.

The Allied AI Agent For This Example

Now that you’ve seen the primary AI agent, let’s take a look at the allied AI agent.

Here it is.

• AI agent routine with tool use and a handoff: Returns And Refunds Agent

“You are a product returns and refund agent for the Widget Corporation.”

“Follow this standard routine:”

“(1) Ask the customer if they want to return the product and get a refund.”

“(2) If the customer says no then go back to Customer Support Agent.”

“(3) Access the WidgetSys database to mark that the product is being returned and the customer will be given a refund.”

“(4) Tell the customer how to return the product and let them know they will be given a refund.”

“(5) Go back to Customer Support Agent and inform that the return and refund processing is now underway.”

End of definition

Once again, the AI agent is defined via the use of natural language.

A handoff back to the primary agent happens in Step #2. Access to the tool WidgetSys takes place at Step #3. Another handoff back to the primary agent occurs in Step #5.

This allied AI agent takes on the task of processing a potential item return and refund. This could have been embedded entirely in the Customer Support Agent, but it turns out to be better for us to make it into a separate routine. Doing so means that we can always make use of the AI agent from other agentic AI that might need to invoke that specific task.

Vital Considerations About These AI Agents

Let’s be contemplative and mindfully explore the big picture. Life is always full of tradeoffs. The use of AI agents is no exception to that rule of thumb. You’ve seen first-hand that a notable plus is the ease of development via natural language.

Time to discuss some of the downsides or qualms.

I provided five steps for the Customer Support Agent and another five steps for the Returns And Refunds Agent. Is that sufficient to cover the wide range of aspects that might arise when successfully performing a customer support role?

Probably not.

Okay, so we might proceed to add more steps. But does that really solve the dilemma of completeness? Probably not. You aren’t likely to lay out all possible steps along with the endless number of permutations and combinations. The generative AI is going to be expected to do the right thing when having to go beyond the stipulated steps.

The generative AI might opt to do something that we would be chagrined or concerned about upon going beyond the stated steps. Keep in mind that the AI is not sentient. It works based on mathematical and computational pattern-matching. Do not expect a kind of human commonsense to be at play, see my analysis at the link here.

Another issue is that everyday words and natural language are said to be semantically ambiguous (see my detailed discussion at the link here). When I told the AI to resolve the customer issue (as part of Step #3 in Customer Support Agent), what does that exactly mean? Resolving something can be a vague concept. The AI could go in many different directions. Some of those directions might be desirable and we would be pleased, while other directions might frustrate a customer and cause poor customer service.

You must also anticipate that the AI could momentarily go off the rails. There are so-called AI hallucinations that generative AI can encounter, see my coverage at the link here. I don’t like the catchphrase because it implies that AI hallucinates in a manner akin to human hallucinations, which is a false anthropomorphizing of AI. In any case, the AI can make up something out of thin air that appears to be sensible but is not factually grounded. Imagine if the AI tells a customer that they can get a refund if they stand on one leg and whoop and holler. Not a good look.

These and other sobering considerations need to be cooked into how you devise the AI agents and how you opt to ensure they operate in a safe and sane manner.

Excerpts Of How OpenAI Explains Swarm

Congratulations, you are now up to speed on the overall gist of agentic AI. You are also encouraged to dig more deeply into Swarm, which is one framework or approach to AI agents. See my coverage at the link here for competing AI agentic frameworks and methods.

Since you are now steeped in some of the agentic AI vocabularies, I have a bit of an informative test or quiz for you. Take a look at these excerpts from the OpenAI blog. I am hoping that you are familiar enough with the above discussion that you can readily discern what the excerpts have to say.

I’m selecting these excerpts from “Orchestrating Agents: Routines and Handoffs” by Ilan Bigio, OpenAI blog, October 9, 2024:

• “The notion of a ‘routine’ is not strictly defined, and instead meant to capture the idea of a set of steps. Concretely, let’s define a routine to be a list of instructions in natural language (which we’ll represent with a system prompt), along with the tools necessary to complete them.”
• “Notice that these instructions contain conditionals much like a state machine or branching in code. LLMs can actually handle these cases quite robustly for small and medium-sized routines, with the added benefit of having ‘soft’ adherence – the LLM can naturally steer the conversation without getting stuck in dead-ends.”
• “Dynamically swapping system instructions and tools may seem daunting. However, if we view ‘routines’ as ‘agents’, then this notion of handoffs allows us to represent these swaps simply – as one agent handing off a conversation to another.”
• “Let’s define a handoff as an agent (or routine) handing off an active conversation to another agent, much like when you get transferred to someone else on a phone call. Except in this case, the agents have complete knowledge of your prior conversation!”
• “As a proof of concept, we’ve packaged these ideas into a sample library called Swarm. It is meant as an example only and should not be directly used in production. However, feel free to take the ideas and code to build your own!”

How did you do?

I had my fingers crossed that the excerpts made abundant sense to you.

Getting Used To Agentic AI

A few final thoughts for now about the rising tide of agentic AI.

Conventional generative AI that you might be using day-to-day tends to do things one step at a time. Agentic AI boosts this by providing potential end-to-end processing for tasks that you might want to have performed on your behalf. Much of the time, agentic AI leans into the capabilities of generative AI.

Lots of AI agents can potentially get big things done.

I am reminded of the famous quote by Isoroku Yamamoto: “The fiercest serpent may be overcome by a swarm of ants.”

Though the bandwagon is definitely toward agentic AI, we need to keep our wits about us and realize that there are strengths and weaknesses involved. Suppose an agentic AI goes wild and like a swarm of bees’ attacks anything within range. Not a good look. All manners of AI ethics and AI law ramifications are going to arise.

You might brazenly assert that a swarm of them will soon emerge.