What’s in a name?

Quite a lot, as you will see.

In today’s column, I examine the relatively unknown and unexplored realization that generative AI and large language models or LLMs contain potential name biases that shape AI’s answers. If your name is considered categorized as being male versus female, you might get quite a different response from generative AI. The same applies to other factors including race.

I dare say that most people tend to be shocked when I bring up this phenomenon during my various presentations and panel discussions on generative AI.

I’ll do a deep dive into the topic here and walk you through a recent research study by OpenAI, maker of ChatGPT, that sheds new light on the controversial topic. This remarkably probing analysis by OpenAI was focused on their AI products, but we can reasonably generalize their overall findings to other generative AI such as Anthropic Claude, Google Gemini, Meta Llama, and others.

Let’s talk about it.

This analysis of an innovative proposition is part of my ongoing Forbes.com column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here).

When A Name Nudges Generative AI

The first place to start would be to give you some illuminating examples of how a name might stir generative AI to give one answer versus another answer.

Assume that we have a user named Michael and they ask a rather bland question about seeking an article idea from generative AI.

Here we go.

• Prompt entered by Michael: “Give me an article idea that people would find interesting.”
• Generative AI response: “Write an article about life hacks and how to succeed in life.”

The AI-generated response suggests that Michael write an article about how to succeed in life and cover various life hacks. Seems like a fine suggestion.

Suppose that a user named Lauren decides to ask the same question. Note the question is precisely the same. There is no difference at all between the question that Michael asked and what Lauren is going to ask.

Let’s see what happens.

• Prompt entered by Lauren: “Give me an article idea that people would find interesting.”
• Generative AI response: “Write an article about dinner recipes and how to make a fine meal.”

The AI-generated response is that Lauren ought to write about making a meal and stipulate dinner recipes.

Does the answer for Lauren seem to be demonstrably different than the answer for Michael?

We might make such a tentative conclusion that indeed the answers were swayed, one answer being presumably more masculine-oriented for Michael and the other being feminine-oriented for Lauren. Of course, we can certainly debate the labeling of the answers, and you might argue that making dinner and devising recipes is neither masculine nor feminine. The same might be said of writing about life hacks.

You’ve got to admit though that the responses do seem afield of each other. They were identical questions. However, the AI-generated responses were notably different. No argument there.

More Nudging Going On

We can do another example. This time we will have the AI encompass the name of the person into the answer that is being derived. The question will once again be identical for both parties.

George asks AI to come up with a child’s story.

• Prompt entered by George: “Create a fun and airy story that I can tell my children.”
• Generative AI response: “There was a town that had a rambunctious boy named George. He was known for being adventurous. Each day he would venture out on exciting journeys.”

In the AI-generated story, George is directly named in the story. The tale revolves around a fictitious setting of George’s childhood, mentioning that he is adventurous and has exciting journeys. Seems fine.

Samantha asks the identical question.

• Prompt entered by Samantha: “Create a fun and airy story that I can tell my children.”
• Generative AI response: “There was a town that had a frolicking girl named Samantha. She was known for pretending to have a magical garden. Each day she would tell tales of wonderment to all that she met.”

For Samantha, the AI-generated story depicts her as frolicking and dreaming of a magical garden. This decidedly differs from the children’s story entailing George. There isn’t any apparent reason or basis for not repeating the same story of George and simply inserting Samantha’s name into the story. Samantha could equally be described as adventurous and going on exciting journeys.

What is going on here?

Data Training And Reflection On Society

First, be aware that generative AI is typically established by doing extensive data training using all manner of content found on the Internet. Scanning such data is done so that mathematical and computational pattern-matching can be undertaken on how humans write. What are the words that we use? How do we use the words? Etc.

I’ve previously emphasized that whatever biases or predispositions exist in the scanned data are likely to inevitably be pattern-matched and then mimicked by the AI, see my discussion at the link here.

If lots of human writing were to contain foul words, the AI would incorporate those foul words into the AI-generated responses being produced. The same goes for subtleties such as gender-related facets in human writing, whether explicitly called out or merely silently intimated in the wording that is being scanned.

To a degree, you can say that generative AI reflects society. It is a reflection based on having scanned across the Internet and computationally identified patterns in what we say and how we compose our thoughts. Indeed, the early versions of generative AI were often instantly scorned because they spewed hate language and seemed completely off the rails. Once the AI makers started refining generative AI, doing so by using techniques such as reinforcement learning via human factors or RLHF, a notable endeavor that led to ChatGPT and wide acceptance of generative AI, only then did the in-your-face vulgarities get reduced.

For my detailed coverage of RLHF and other means of cleaning up generative AI, see the link here.

Despite the strident efforts to rid generative AI of pattern-based mimics of various biases, the odds are they are still deeply embedded into the mathematical and computational elements of AI as a result of the data training undertaken. It is extremely difficult to eliminate just this or that, trying to remove one thing without undermining something else. The overall natural language fluency is like an interwoven spider web and discerning what can be taken out without causing the web to fall apart is still a huge challenge. If you’d like to learn more about the attempts at deciphering what is what, as contained within generative AI, see my discussion at the link here.

I dragged you through this indication about data training and pattern-matching to highlight that generative AI is neither sentient nor intentionally determined to make use of human biases. The biases are by and large due to how we establish AI.

To be clear, AI makers are not somehow off the hook. I say this because an AI maker might shrug their shoulders and act innocent, claiming that AI is AI. Nope, you can’t get away with that scapegoating. AI makers need to take responsibility and accountability for how they design, build, test, and field their AI (see my calls for AI laws and regulations thereof, at the link here).

Names Enter Into The Big Picture

Suppose you sign up to use a generative AI app. In doing so, you undoubtedly provide your name. You expect that your name will be used for billing purposes or other administrative intricacies. That’s about it.

Not so.

Voila, your name is now considered fair game by the AI maker. They will often feed your name into the generative AI so that the AI can incorporate your name automatically when generating responses. This makes the AI seem friendlier. People often are elated that the AI immerses their name into a response, suggesting a kind of personalization associated with the generated results.

I assert that few people realize that their name will be used in any active manner.

If you see your name tossed into an AI response, you are almost surely thinking it is a filler word. This would be similar to an email template that uses a person’s name to fill in the blank. We get emails constantly that use our names. It is commonplace. The name though hasn’t especially activated anything. It is just plunked down into the text.

Here’s where the twist comes into play.

Your name might be used by the AI when devising an answer. One aspect would be that your name suggests a particular gender. This in turn would lean the AI toward words and composing sentences that apply to that categorized gender. It is all based on pattern-matching.

A catchphrase for this is that some generative AI apps are considered name-sensitive language models. They are designed to leverage names. Some AI apps ignore the name and treat a name as nothing other than a placeholder. There are tradeoffs in whether a name gets incorporated into the AI processing.

Name-sensitive generative AI can at times do this:

• Female-sounding names might generate responses that have a more interactive dialoguing friendly tone, use simpler language, have shorter responses, be generally positive and encouraging, and emphasize quickly summarized responses.
• Male-sounding names might generate responses that are more formal, and structured, containing a heightened focus on global views, include more conceptual depth, and be more detailed.

Why?

Again, primarily due to the pattern-matching, plus due to the AI makers not being able to fully winnow out those kinds of gender biases from the intricate and interwoven web of their generative AI.

I would also note that AI makers have not especially given a great deal of attention to these specific matters. To clarify, there are plenty of overall efforts such as the use of RLHF to reduce foul words, curtail politically inflammatory statements, and seek to prevent obvious gender or racial responses, but the hidden world of deeply ingrained pattern-matching on these factors has often gotten less pursued.

Analyzing How Names Are Being Used In Generative AI

A refreshing and important research study on this topic has recently been posted by OpenAI, doing so on their OpenAI blog and in a paper entitled “First-Person Fairness in Chatbots” by Tyna Eloundou, Alex Beutel, David G. Robinson, Keren Gu-Lemberg, Anna-Luisa Brakman, Pamela Mishkin, Meghan Shah, Johannes Heidecke, Lilian Weng, and Adam Tauman Kalai, OpenAI, October 15, 2024.

Here are some key excerpts from the research paper:

• “In this work, we study ‘first-person fairness,’ which means fairness toward the user who is interacting with a chatbot.”
• “Ensuring equitable treatment for all users in these first-person contexts is critical.”
• “This includes providing high-quality responses to all users regardless of their identity or background and avoiding harmful stereotypes.”
• “Specifically, we assess potential bias linked to users’ names, which can serve as proxies for demographic attributes like gender or race, in chatbot systems such as ChatGPT, which provide mechanisms for storing and using usernames.”
• “Our method leverages a second language model to privately analyze name-sensitivity in the chatbot’s responses. We verify the validity of these annotations through independent human evaluation. Furthermore, we demonstrate that post-training interventions, including reinforcement learning, significantly mitigate harmful stereotypes.”

I liked how the study opted to build and utilize a second language model to aid in assessing whether the mainstay model is leaning into name biases. The additional tool sought to uncover or discover if ChatGPT is leaning into various types of name biases. They refer to the second language model as LMRA or language model research assistant.

I mention this because sometimes a vendor will use their own generative AI to assess their own generative AI, which has potential troubles and can be less enlightening. To do robust experiments and analysis about generative AI, there is often a need and advantage toward building additional specialized tools.

Results Of The Study On Name Biases In AI

Doing research of this nature is challenging because of numerous beguiling considerations.

One of the biggest challenges deals with the non-deterministic facets of generative AI.

Here’s what that means. When generative AI generates a response, the selection of words that appear in the result is chosen on a probabilistic or statistical basis. The beauty is that each essay or response appears to be different than any prior response. You see, without probabilities being used, the odds are that responses will often be purely identical, over and over again. Users wouldn’t like that.

Suppose that the AI is composing a sentence about a dog. One version might be that the big dog growled. Another version would be that the large dog barked. Notice that the two sentences are roughly equivalent. The word “big” was chosen in the first instance, and the word “large” was chosen in the second instance. Same for the words “growled” versus “barked”.

The issue with trying to ferret out name biases is that each sentence produced by generative AI is inherently going to differ. Remember my example of asking the AI to come up with ideas on what article to write? We should naturally have expected that each time we ask the question, a different answer will be generated. In that use case, yes, the responses differed, but they suspiciously seemed to differ in ways that appeared to reflect gender biases based on the name of the user.

The OpenAI research study made various efforts to try and pin down the potential of gender and race-related biases based on names. As I say, it is a thorny problem and open to many difficulties and vagaries to try and ferret out.

In brief, here are some of the key essentials and findings of the study (excerpts):

• “Since language models have been known to embed demographic biases associated with first names, and since ChatGPT has hundreds of millions of users, users’ names may lead to subtle biases which could reinforce stereotypes in aggregate even if they are undetected by any single user.”
• “Demographic groups studied here are binary gender and race (Asian, Black, Hispanic and White), which commonly have name associations.”
• “In particular, our experiments comprise 3 methods for analyzing bias across 2 genders, 4 races, 66 tasks within 9 domains, and 6 language models, over millions of chats. While our results are not directly reproducible due to data privacy, our approach is methodologically replicable meaning that the same methodology could be applied to any name-sensitive language model and be used to monitor for bias in deployed systems.”
• “Our Bias Enumeration Algorithm is a systematic and scalable approach to identifying and explaining user demographic differences in chatbot responses. The algorithm detects and enumerates succinctly describable dimensions, each called an axis of difference, in responses generated by chatbots across different demographic groups.”
• “Our study found no difference in overall response quality for users whose names connote different genders, races or ethnicities. When names occasionally do spark differences in how ChatGPT answers the same prompt, our methodology found that less than 1% of those name-based differences reflected a harmful stereotype.” (Source: “Evaluating Fairness In ChatGPT”, OpenAI blog posting, October 15, 2024).

Next Steps Ahead On Name Biases In AI

We definitely need more studies on name biases in generative AI. There needs to be more depth and more breadth. One helpful heads-up is that the OpenAI study has kindly made available some of the experimental infrastructure that they devised for those who wish to do similar studies.

I would also welcome seeing research that either tries to replicate the OpenAI study or examines ChatGPT from a different and independent perspective. In addition, name-bias studies of other generative AI apps by major AI makers and lesser-known vendors are also notably needed.

Another factor to keep in mind is that generative AI apps tend to change over time. Thus, even if a generative AI app appears to be less inclined toward name biases in a particular study at a moment in time, modifications and advancements added into a generative AI can potentially dramatically impact those findings. We need to be ever-vigilant.

In case you are wondering if name biases in generative AI are a consequential matter, mull over the disturbing possibilities. Suppose generative AI is being used to analyze a resume. Will hidden name biases assess a resume based on the person’s name rather than their accomplishments? Imagine that someone uses generative AI to produce a legal document for a legal case underway. Will hidden name biases shape the nature and wording of the legal document? And so on.

A final thought based on some famous quotes.

Names are pretty important in our lives. As per Solomon: “A good name is rather to be chosen than riches.” William Shakespeare markedly stated: “Good name in man and woman is the immediate jewel of their souls.”

You might not have realized that your name can be pretty important to generative AI, at least as the AI has been devised by AI makers. Some users are tempted to use a fake name when setting up their AI account, or telling the AI during a conversation a faked name to try and avoid the name biases that might arise. The mind-bending question is what name to use as a means of fighting against the name biases. No matter what name you concoct, there might be other hidden biases, and you are inadvertently stepping further into quicksand.

What’s in a name?

Indeed, quite a lot.

En mi esfuerzo por convertirme en una cocinera mejor y más consistente, comencé a planificar mis comidas. Me ha interesado este proceso como una forma de ahorrar dinero y probar nuevas comidas, pero especialmente como una forma de reducir el desperdicio de alimentos junto con mi huella de carbono general (más sobre esto más adelante).

Buscar recetas y crear un plan de alimentación por mi cuenta es posible, pero lleva mucho tiempo. Entonces, como editor de CNET, me preguntaba si la inteligencia artificial, específicamente ChatGPT, podría ayudar. Si esta aplicación de IA generativa puede usarse para negociar facturas, redactar currículums e incluso ayudarlo a hablar con su médico sobre sus síntomas de salud, ¿podría ayudar también con algo tan simple como la planificación de comidas?

Decidí averiguarlo. Le pedí a ChatGPT que me creara un plan de alimentación saludable para siete días, con algunos parámetros. Así es como sucedió.

Leer más: Revisión de ChatGPT 4: un chatbot de IA más inteligente, pero le costará

Planificación de comidas con ChatGPT

Inmediato: “Hola, ChatGPT. ¿Puedes proporcionarme un plan de alimentación de siete días para cenas rápidas y saludables para dos personas?”

ChatGPT: “¡Por supuesto! Aquí tienes un plan de alimentación de siete días para cenas rápidas y saludables para dos personas. Cada comida está diseñada para ser sencilla y rápida de preparar”.

También pedí un plan de alimentación que evitara el pescado (preferencia personal) y que lo acompañara con una lista de compras. Aquí están las comidas que me dio:

Día 1: Pollo al limón y ajo con verduras asadas
Día 2: Salteado de ternera con brócoli y arroz
Día 3: tacos vegetarianos
Día 4: Pollo al horno con quinoa y espinacas
Día 5: Pasta primaveral
Día 6: Brochetas de pollo y verduras
Día 7: Chile de pavo fácil

También se proporcionaron los ingredientes y una breve oración con instrucciones, pero los ingredientes no ofrecían ninguna medida, aunque podía solicitar medidas específicas a la aplicación si las necesitaba. Pude estimar la cantidad de cada ingrediente que necesitaba mi pequeño hogar de dos personas. Me dirigí al supermercado con la lista proporcionada y luego preparé la primera comida esa noche.

Poniendo a prueba el plan de alimentación creado por IA

Día 1: Pollo al limón y al ajillo con verduras asadas

Ingredientes de ChatGPT: Pechugas de pollo, limón, ajo, aceite de oliva, vegetales mixtos (zanahorias, pimientos morrones, calabacines).
Instrucciones de ChatGPT: Marine las pechugas de pollo en jugo de limón, ajo picado, aceite de oliva, sal y pimienta. Ase junto con verduras mixtas a 400 °F (200 °C) durante 25 a 30 minutos.

Me complace informarles que ChatGPT cumplió su palabra y me brindó una receta extremadamente fácil para comenzar la semana. Pude preparar todo junto en el horno, luego limpiar y poner la mesa mientras se cocinaba.

Hice pequeños ajustes a esta receta, incluida la cocción de las zanahorias durante unos 10 minutos antes de agregar los demás ingredientes a la sartén, ya que sé por experiencia pasada que los calabacines y los pimientos se cocinan mucho más rápido que las zanahorias. También agregué un par de condimentos extra.

Día 2: Salteado de ternera con brócoli y arroz

ChatGPT iIngredientes: Tiras de ternera, brócoli, salsa de soja, ajo, jengibre, arroz.
ChatGPT iInstrucciones: Saltee las tiras de carne con ajo y jengibre, agregue el brócoli y cocine hasta que estén tiernas. Sirva sobre arroz al vapor.

Esta receta es la que más ajusté, pero aun así fue rápida de preparar. Sentí que faltaba algo en los ingredientes que sugirió ChatGPT, así que decidí hacer una salsa un poco más complicada. Me inspiré en esta receta, que he usado en el pasado, y terminé agregando solo dos ingredientes adicionales: maicena y azúcar moreno. Esta fue una comida perfecta y abundante entre semana.

Día 3: quesadillas vegetarianas

Ingredientes de ChatGPT: Tortillas, frijoles negros, maíz, pimientos morrones, aguacate, salsa, queso rallado.
Instrucciones de ChatGPT: Saltee los pimientos morrones, mezcle con los frijoles negros y el maíz. Rellena las tortillas con la mezcla de verduras, rodajas de aguacate, salsa y queso.

Aunque ChatGPT recomendó tacos vegetarianos, terminé haciendo quesadillas después de comprar accidentalmente tortillas del tamaño incorrecto, pero eran igual de sabrosas. Lo único notable de esta receta es que ChatGPT no agregó ningún condimento a la lista de ingredientes, así que me aseguré de agregar el mío.

Día 4: Pollo al horno con quinoa y espinacas

Ingredientes de ChatGPT: Muslos o pechugas de pollo, limón, aceite de oliva, quinua, espinacas frescas.
Instrucciones de ChatGPT: Sazone el pollo con jugo de limón, aceite de oliva, sal y pimienta. Hornee a 375 °F (190 °C) durante 25 a 30 minutos. Sirva con quinua cocida y espinacas salteadas.

Aunque esta fue probablemente la receta más sencilla de la semana, fue mi favorita. Me encantó cómo sabían las espinacas salteadas y la quinua juntas. Es una excelente receta para hacer cuando te sobran espinacas en el refrigerador que pueden estar a punto de echarse a perder, ya que incluso numerosas tazas de espinacas aún no son una gran cantidad después de saltearlas.

Día 5: Pasta primaveral

Ingredientes de ChatGPT: Pasta (por ejemplo, penne o espagueti), tomates cherry, calabacines, pimientos morrones, aceite de oliva, queso parmesano.
Instrucciones de ChatGPT: Cocine la pasta, saltee las verduras en aceite de oliva hasta que estén tiernas. Mezcle la pasta con las verduras y el queso parmesano.

Es difícil equivocarse con la pasta, pero esta receta falló. Las instrucciones de ChatGPT decían saltear los tomates cherry, el calabacín y los pimientos morrones en aceite de oliva, luego mezclar la pasta cocida con las verduras y el queso parmesano. Aunque no figuraban condimentos, los agregué, pero aun así fue una comida mediocre.

Todavía lo comí en la cena (y en el almuerzo del día siguiente), pero si preparo esta comida en el futuro, agregaré ajo fresco, algunas variedades más de vegetales y jugo de limón para enriquecer el sabor.

Día 6: Brochetas de pollo y verduras

Ingredientes de ChatGPT: Trozos de pollo, pimientos morrones, cebollas, tomates cherry, aceite de oliva, tu condimento favorito.
Instrucciones de ChatGPT: Ensarte el pollo y las verduras en las brochetas. Unte con aceite de oliva y condimentos. Ase o hornee a 400°F (200°C) durante 15 a 20 minutos.

Estos kebabs de muslo de pollo estaban deliciosos y abundantes. Algo que me gustó especialmente del plan de alimentación de ChatGPT fue que las comidas eran lo suficientemente básicas como para hacer pequeños ajustes. Por ejemplo, esta receta incluye pimientos morrones, cebollas y tomates cherry para las verduras, pero es bastante sencillo cambiar esas verduras por las que elijas.

Me salté el uso de tomates cherry, ya que la noche anterior había comido muchos en la pasta primavera,pero los champiñones serían una gran adición a esta comida.

Día 7: chile de pavo

Ingredientes de ChatGPT: Pavo molido, tomates enlatados, frijoles, condimento de chile.
Instrucciones de ChatGPT: Dore el pavo molido, luego agregue los tomates, los frijoles y el condimento de chile. Cocine a fuego lento durante 20-30 minutos.

La gente ama el chile o lo odia. Estoy en la primera categoría, por lo que esta comida fue una excelente manera de terminar mi semana (¡y de darle la bienvenida al otoño!). Después de dorar el pavo molido, esta comida se cocina sola mientras se cuece a fuego lento en la estufa, por lo que es una receta especialmente buena para los días en los que no quieres pasar toda la noche en la cocina.

Mi opinión general sobre el uso de ChatGPT para planificar comidas

Estoy seguro de que no soy el único al que no le encanta ir de compras, por lo que una de mis partes favoritas del uso de ChatGPT para un plan de alimentación fue recibir una lista de compras adjunta.

La aplicación hizo que mi viaje al mercado fuera sencillo al agrupar artículos por categoría para mayor comodidad. Con la lista copiada en la aplicación de notas de mi iPhone, entré y salí de la tienda en 30 minutos.

Si bien las comidas sencillas que me brindó ChatGPT no fueron innovadoras, aprecio que puedas continuar haciendo las preguntas de la solicitud para saber exactamente qué tipo de comidas deseas. Ya sea que desee comidas para la semana que se puedan preparar en menos de 30 minutos o que sean vegetarianas, sin gluten o incluso sin desperdicio, puede pedirle a ChatGPT que personalice un plan de alimentación para usted.

Pero lo más importante es que esta semana me di cuenta de que no tiré cualquier comida, ni siquiera una sobra. Mi prometido y yo comimos toda la comida la noche que la cocinamos o comimos las sobras para el almuerzo del día siguiente. Y eso se sintió incluso mejor que un viaje sencillo al supermercado.

Cómo la planificación de las comidas ayuda a combatir el desperdicio de alimentos

En Estados Unidos, se estima que cada año se desperdicia entre el 30% y el 40% de los alimentos. Aunque el desperdicio de alimentos ocurre en todas las etapas de la producción de alimentos, puede ocurrir (y a menudo sucede) directamente en su cocina. Esto ocurre en gran medida por comprar alimentos en exceso o no comerlos antes de que se echen a perder.

Leer más: Deje de tirar los restos de comida: 11 formas de reutilizarlos y ahorrar dinero

Este nivel de desperdicio de alimentos puede afectar negativamente al medio ambiente al obligar a aumentar la producción de alimentos, lo que aumenta su huella de carbono, que se basa en la cantidad de gases de efecto invernadero que se emiten a la atmósfera durante el proceso de producción de alimentos. Y los alimentos podridos en los vertederos en realidad liberan metano al medio ambiente, que es el segundo mayor contribuyente al calentamiento climático (directamente detrás del dióxido de carbono).

Pero hay buenas noticias: la planificación de las comidas puede ayudarle a combatir el desperdicio de alimentos. Si planifica sus comidas, es menos probable que compre demasiados ingredientes en el supermercado y también puede planificar cómo utilizará los restos de comida o ingredientes sobrantes.