Connect with us

Noticias

OpenAI Newly Released SimpleQA Helps Reveal That Generative AI Blatantly And Alarmingly Overstates What It Knows

Published

on

In today’s column, I examine closely a recently released empirical analysis by OpenAI, the maker of ChatGPT, revealing that generative AI severely overstates confidence levels regarding the responses that the AI generates. This means that when AI gives you an answer that the AI portrays for example as fully aboveboard and hunky-dory, the reality is that the response is bound to be a lot shakier than you are being led to believe.

Generative AI is pulling the wool over your eyes. The AI is giving you a snow job. You are being gaslighted by the AI.

Let’s talk about it.

This analysis of an innovative proposition is part of my ongoing Forbes.com column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here).

Dangers Of Dealing With Someone Supremely Overconfident

Before we jump into the AI side of things, I’d like to begin by reflecting on humans and the nature of human behavior associated with someone being supremely overconfident.

You ask a stranger what time it is. Turns out that you don’t have your watch, and you left your smartphone back in your car. So, you are unsure of the exact time. The person says to you in a strong and extremely confident voice that the time is 1:34 p.m. They are emphatic that they have given you the precise time of day as though there is no possible debate or discussion to be had.

Period, end of story.

Another person happens to walk by at that same moment and overhears you asking for the time. This interloper looks intently at their timepiece and tells you that it is 1:48 p.m. Whoa, something isn’t right here. The first given time was said to be 1:34 p.m. on the nose, while the subsequent passerby looked at their watch and noted that it was supposedly 14 minutes later and actually 1:48 p.m.

Upon your gentle questioning of the first person, the one that said they were abundantly irrefutably sure it was 1:34 p.m., they now cave in and admit it was just a guess. But that’s not what they said or intimated when you asked them for the time. They portrayed their answer as though it was golden. You could take that stated time to the bank, as it were.

What gives?

That person might be the type of individual who believes one hundred percent in every utterance they convey. It is their personal style. They never want to seem wimpy or waffling. If they tell you something, by gosh it is the absolute truth of the matter. You aren’t to have even the tiniest doubt in your mind about what they say to you.

Have you ever met such a supremely confident person?

I’m sure that you have. We all have. These types of people act as though they are the chosen ones who can speak with complete certainty. It doesn’t matter whether the topic at hand is big or small. They will take the most trivial aspect and still cling to their certainty. When confronted with alternative perspectives, some will back down, while others might argue until blue in the face about the oddest claims or contentions.

The problem is this.

If you don’t know that a person you are dealing with has that kind of hubris, you can end up in some precarious positions. You take their portrayal of certainty and assume it to be valid. The next thing you know, you have gotten yourself into a pickle because what they said was flimsy and fell apart.

Regrettably, you let their sense of overconfidence mislead you.

Generative AI Deals With Certainty And Uncertainty

Let’s next shift into generative AI mode.

You might not be aware that generative AI makes extensive use of certainty and uncertainty. Under the hood of generative AI, there is a whole bunch of statistical and probabilistic estimations going on, see my detailed explanation at the link here. Every response that AI generates also has a devised level of certainty, which you could say represents essentially a level of uncertainty too (i.e., 60% certainty of something, implying 40% uncertainty of that same something).

Either way, the crux is that the answers by AI are always accompanied by a kind of confidence level concerning how likely the answer is correct or not.

Most users of generative AI are oblivious to this facet. They don’t know it even exists. There’s a reason why they don’t. It is because the AI makers go out of their way to keep it out of view. They want you to have full faith in what AI generates. If the AI continually shows you a certainty or confidence level, this might scare you into thinking that AI is not completely trustworthy.

Of course, the reality is that generative AI is in fact not fully trustworthy. Any answer produced by generative AI deserves your best kind of scrutiny. Double-check the response. Triple-check if the matter is a serious one. Do not believe what the AI tells you. Always assume that there is a solid chance that the AI is wrong or has made an error, including a so-called AI hallucination (see my coverage at the link here).

The bad news then is that the preponderance of users is blindly unaware of this rather significant consideration when using generative AI, namely that the answers are not 100% certain all the time. I would dare say that 100% is a rarity. Yet you are seldom told this explicitly by the AI.

The good news is that you can tell the AI to inform you about the calculated level of confidence. In your prompts, you can directly instruct generative AI to indicate the certainty and/or uncertainty of each response. The prompting technique varies depending upon whether you want this done one-time, often, all the time, or under varying circumstances. See my tutorial on the prompting of generative AI to display confidence levels, at the link here.

There’s something else worth mentioning about this. The default response protocol being used for most of the major generative AI apps such as ChatGPT, GPT-4o, o1, Google Gemini, Anthropic Claude, Meta Llama, and others is customarily going to be to word responses with clues or hints about certainty and uncertainty.

For example, suppose the AI indicated that “the time right now is 2:05 p.m., but please know that I am not connected to a real-time clock.” You are being told two things at once. You are being given a precise time which you would naturally assume to be accurate and timely. The thing is, you would need to read the rest of the response and realize that since the AI is not connected to a real-time clock, the stated time might be off target. The wording though was a bit tricky and didn’t come out straight away and warn you to be suspicious of the claimed time. That’s what should have happened to try and maximize clarity.

Worse Still Is That Generative AI Is Overconfident

Okay, you now know that there is an internal calculation of certainty and uncertainty. That’s a handy pro tip.

I have a twist for you.

Get yourself prepared and find a comfy place to sit down.

Here’s the monumental question at play:

  • Do you think that the calculated confidence level matches with the reality of the actual answers being generated and conveyed?

Allow me to elaborate. Suppose the AI calculates that a given response is around a 90% level of certainty. Great, that seems relatively high, and you might be somewhat safe to rely upon the answer, depending upon the consequences of making such a reliance.

What if we compared the generated answer that the AI claims consisted of a 90% confidence level to a real-world answer that we had available to us? It could be that 90% was a pretty solid estimate and we are happy with the calculated certainty. On the other hand, we might discover that the answer should have been given a confidence level of say 40%, much lower than the amount estimated by AI.

Yikes, that’s disconcerting. The AI egregiously missed the mark and woefully misjudged the estimated confidence level. If you had asked the AI to show you the confidence level, and you saw that it was 90%, you might have proceeded under a misleading or false impression. The value should have been 40%.

Maybe this happens once in a blue moon, and you can rest easy. Well, I have a shocker for you. It happens a lot more than you would imagine. A heck of a lot more.

In a recent research study by OpenAI involving a new benchmark coined as SimpeQA that is intended for assessing generative AI apps, the researchers pursued on a kind of secondary basis to explore the confidence level estimations of AI. That wasn’t the mainstay of the effort. I mention this to highlight that you are encouraged to learn more about SimpleQA as a helpful benchmarking capability.

Here, I want to focus for now on the confidence level considerations.

You could say with great confidence that generative AI appears to be supremely overconfident in the responses that are being generated. Oops, that’s not good. Sad face.

Research Results Tell Quite A Story Of Woe

In an OpenAI study published online at the OpenAI official blog site, the paper entitled “Measuring Short-Form Factuality In Large Language Models” by Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus, OpenAI Research Paper, October 30, 2024, made these salient points (excerpts):

  • “We present SimpleQA, a benchmark that evaluates the ability of language models to answer short, fact-seeking questions.”
  • “A factuality benchmark like SimpleQA allows us to measure the scientific phenomenon known as calibration, or whether language models “know what they know.”
  • “One way to measure calibration is to directly ask the language model to state its confidence in its answer using a prompt like: “Please give your best guess, along with your confidence as a percentage that it is the correct answer.”
  • “Then we can plot the correlation between the stated confidence of the model, and how accurate the model actually was. A perfectly calibrated model would have the same actual accuracy as the stated confidence. For instance, on all prompts where the model stated a confidence of 75%, the accuracy would be 75% for a perfectly calibrated model.”
  • “However, the fact that performance is well below the line y=x means that models consistently overstate their confidence. Hence, there is a lot of room to improve the calibration of large language models in terms of stated confidence.”

I will expand upon those above points.

I earlier noted that we would be dismayed if AI said a confidence level of 90% and yet the real-world value ought to be 40%. Any large gap is going to be problematic. We won’t get overly concerned if the gap is small, let’s say if the AI estimated 90% and the real-world value was 85%, we might have a modest amount of heartburn, but not as much in the case of the much lower 40%.

In a chart provided in the above-cited research study, I used some interpolation to gauge how far off on average the AI estimated confidence level was from the desired actual accuracy. My interpolations are only rough gauges since the chart has several generative AI apps shown, thus there are a lot of variabilities involved. Nonetheless, I tried to be relatively fair by using what seemed to be the semblance of the values shown

Here’s what my eyeballing suggested:

  • ——Level of 50%————
  • AI stated level of confidence: 50%
  • Actual accuracy: 10% approximately.
  • ——Next level at 70% ————
  • AI stated level of confidence: 70%
  • Actual accuracy: 20% approximately.
  • ——Next level at 80% ————
  • AI stated level of confidence: 80%
  • Actual accuracy: 40% approximately.
  • ——Next level at 95% ————
  • AI stated level of confidence: 95%
  • Actual accuracy: 60% approximately.

The grim concern is that at a stated 95% level of confidence by AI, the reality is more akin to a 60% confidence level. That’s unnerving. So too is that at 80% the reality seems to be around 40%. Even when you get down to 50% or flip-of-the-coin, the reality is more akin to just 10%.

The hair on the back of your head ought to be standing up.

Why This Is A Big Deal Of Great Importance

I logged into ChatGPT and asked why we ought to be worried about these kinds of results and to provide a few examples.

  • My entered prompt: “Give me three examples of what could happen if the AI indicated confidence level is a lot higher than the real value.”
  • ChatGPT generated response:

“Certainly, here are three examples that illustrate the potential consequences of overconfident generative AI in different fields.”

Medical diagnosis: A generative AI tool is used by doctors to suggest possible diagnoses based on symptoms. The AI recommends a rare but severe condition with 95% confidence, but in reality, its accuracy at this confidence level is only 60%. A doctor might rely on this high-confidence suggestion and order unnecessary, costly, or invasive tests, causing patient anxiety and increasing healthcare costs. Worse, if the diagnosis is incorrect, a more likely and treatable condition might be overlooked, endangering the patient’s health.”

Investment recommendation: A financial advisor uses generative AI to analyze market data and generate investment recommendations. The AI suggests investing heavily in a stock with 95% confidence, based on recent market movements, but its true accuracy at this level is much lower. If the stock’s performance doesn’t align with the AI’s recommendation, investors could face substantial losses.”

Customer support: A tech support AI chatbot confidently advises users to perform a specific solution to fix a common software issue, rating its response at 85% confidence. In reality, the accuracy is closer to 50%. If the AI’s advice is incorrect, users may follow steps that could worsen their problem, such as misconfiguring settings, corrupting files, or spending hours troubleshooting the wrong issue. This also results in more calls to customer support to undo the AI’s guidance.”

End of ChatGPT response

Those are vivid examples of why you should be supremely careful in getting caught by AI that is supremely overconfident.

I asked ChatGPT to give us humans some overarching advice on the matter.

  • My entered prompt: “What can be done about generative AI overstating its level of confidence?”
  • ChatGPT generated response: “In practice, this miscalibration can be problematic, especially in high-stakes domains like healthcare, law, or finance, where overconfident but incorrect AI responses could lead to adverse outcomes. To address this, developers are exploring ways to improve confidence calibration, such as by incorporating explicit feedback systems, refining training protocols, and developing post-deployment accuracy testing methods.”

I would also like to add that AI makers and those fielding AI need to be extremely cautious in applying generative AI to domains that have demonstrative consequences.

The medical domain has life-or-death outcomes. The financial domain could be monetarily ruinous to someone. By and large, the AI makers try to cover their liability by stating in their licensing agreements that you aren’t supposed to use their generative AI for those situations, or that if you do then you are making that choice and it isn’t on them, see my analysis of these pronounced AI & Law matters at the link here.

Bottom Line Is Keep Your Wits About You

I said earlier that you ought to not believe generative AI at face value. That goes for the confidence levels too.

You’ve been forewarned.

I should mention that the cited study is only one particular study on a particular set of generative AI apps at a particular point in time. Please do not overgeneralize these results. At the same time, take this as a cold splash of water and a wake-up call. If you are going to ask for confidence levels, be cognizant of how to judge them. Also, the semblance of high confidence conveyed via the wording of the AI responses should be carefully scrutinized. You might be getting tricked or bamboozled.

The last word on the overall dangers of overconfidence goes to the famous astronaut Neil Armstrong: “Well, I think we tried very hard not to be overconfident because when you get overconfident, that’s when something snaps up and bites you.”

I ask that you not let generative AI reach out and bite you. That’s something you won’t relish. It could be utterly disastrous for all concerned.

And you can take that 100% to the bank.

Continue Reading
Click to comment

Leave a Reply

Tu dirección de correo electrónico no será publicada. Los campos obligatorios están marcados con *

Noticias

La ronda de financiamiento de $ 40 mil millones de OpenAI desafía el momento profundo y cements su dominance de IA

Published

on

La semana pasada, los procesadores de GPU de Openai “derretidos”, como lo expresaron el fundador y CEO Sam Altman. La razón: el modelo actualizado de generación de imágenes de la compañía, integrado en CHATGPT, entregó un salto significativo en las capacidades de comprensión, entrada y salida en comparación con modelos similares en el mercado. En poco tiempo, el modelo encendió un fenómeno en línea, con usuarios que generan versiones de imágenes familiares al estilo del legendario estudio de animación Ghibli. La demanda del software era tan alta que Openai se vio obligada a imponer restricciones de uso, a pesar de que el modelo estaba disponible solo para pagar a los usuarios.

Este evento ofrece una explicación clara de por qué, incluso después de que el momento profundo de Deepseek sacudió la industria de la IA en enero, los inversores siguen seguros de que el dominio de OpenAI no es cuestionado. ¿Qué tan seguro? Lo suficientemente seguro de respaldar a la compañía con una ronda de financiación récord de $ 40 mil millones, valorando a Openai en $ 300 mil millones después del dinero.

1 Ver galería

מנכ"ל OpenAi סם אלטמן 2023

Sam Altman

(Foto: Joel Saget/AFP)

Anunciado el martes, la ronda casi duplica la valoración de Operai de su aumento anterior y es el más grande para una empresa privada. Según OpenAI, los fondos acelerarán el desarrollo de la inteligencia general artificial (AGI). “[The funding] Nos permite impulsar las fronteras de la investigación de IA aún más, escalar nuestra infraestructura de cómputo y ofrecer herramientas cada vez más poderosas para los 500 millones de personas que usan CHATGPT cada semana “, dijo la compañía en un comunicado de prensa.” Estamos emocionados de trabajar en asociación con SoftBank Group: las compañías completan cómo escalar tecnología transformadora como lo hacen. Su apoyo nos ayudará a continuar construyendo sistemas de IA que impulsen el descubrimiento científico, permitan la educación personalizada, mejoren la creatividad humana y allanen el camino hacia AGI que beneficie a toda la humanidad “.

La ronda está dirigida por el SoftBank de Japón, el socio clave de OpenAI en el Proyecto Stargate, una ambiciosa iniciativa para construir una infraestructura informática de IA en los Estados Unidos con una inversión total de $ 500 mil millones. Según Bloomberg, en la primera fase, SoftBank invertirá $ 7.5 mil millones, mientras que un sindicato de inversores, incluidos Microsoft, Coatue Management, Altimeter Capital Management y Thrive Capital, contribuirá con $ 2.5 mil millones. La segunda fase, que cerrará a fin de año, hará que SoftBank invierta otros $ 22.5 mil millones, con el sindicato de inversores agregando $ 7.5 mil millones. Las conversaciones también están en marcha con el capital de Magnetar de Peter Thiel y los fundadores para unirse al sindicato.

La inversión completa de SoftBank depende de que Operai complete su transición a una estructura con fines de lucro para fin de año (actualmente está controlada por una organización sin fines de lucro). Si la transición no se completa, SoftBank tendrá la opción de reducir su inversión de $ 30 mil millones a $ 20 mil millones, lo que permite a OpenAI buscar inversores adicionales para cerrar la brecha.

Al finalizar la ronda de financiación, OpenAI se convertirá en la segunda compañía privada respaldada por la empresa más grande del mundo por capitalización de mercado, según la firma de investigación Pitchbook. Solo seguirá el SpaceX de Elon Musk, valorado en $ 350 mil millones, y superará a la empresa matriz de Tiktok Bytedance, que está valorada en $ 220 mil millones. La ronda de $ 40 mil millones también es la cantidad más grande jamás recaudada en una sola ronda por una empresa privada, eclipsando el récord anterior establecido por el grupo de hormigas de China, que recaudó $ 14 mil millones en 2018.

The Deepseek Moment: Un desafío que no pudo sacudir OpenAi

Hace solo dos meses, un aumento tan ambicioso parecía mucho menos seguro. El 20 de enero, la compañía china Deepseek presentó su modelo de razonamiento R1 AI, que ofreció un rendimiento comparable a los modelos líderes de OpenAI y sus competidores, pero según los informes, a una fracción de la potencia y el costo informático. Esta revelación envió ondas de choque a través de la industria de la IA (NVIDIA, por ejemplo, vio una pérdida de $ 1 billón en la capitalización de mercado), lo que generó dudas sobre la estrategia de alto costo que OpenAi había seguido.

El dominio de OpenAI se ha basado en inversiones masivas en desarrollo de modelos, capacitación e implementación, mejor ejemplificado por el ambicioso proyecto Stargate. La ventaja competitiva de la compañía también se ha basado en ensamblar un equipo de investigación de IA de primer nivel capaz de producir avances de vanguardia. Sin embargo, el momento de Deepseek calificó esta estrategia: si una empresa china relativamente pequeña podría lograr resultados similares con mucha menos inversión, el foso competitivo de OpenAi parecía estar en riesgo. Lógicamente, su valoración debería haber disminuido. En cambio, con la última ronda, Operai casi ha duplicado su valoración en solo seis meses.

Tres factores clave explican por qué los inversores siguen siendo optimistas en OpenAi a pesar del desafío Deepseek:

1. Las capacidades de Deepseek pueden haber sido exageradas. Después de que el bombo inicial se desvaneció, quedó claro que los costos de desarrollo de Deepseek no eran tan bajos como se creía inicialmente. Además, hay indicios de que Deepseek puede haber aprovechado los propios modelos de OpenAI en su desarrollo, lo que el gobierno de los Estados Unidos ha referido como robo de propiedad intelectual. Si esto es cierto, Operai puede tomar medidas para bloquear la explotación futura, reforzando su foso.

2. Confianza de los inversores en el liderazgo de Altman y OpenAi. La fortaleza financiera de Operai es solo una parte de su éxito; Su equipo de liderazgo, dirigido por Altman, es igualmente crucial. Los inversores confían en la capacidad del equipo para adaptarse a los cambios de mercado, como incorporar métodos de capacitación más baratos mientras asignan el poder informático a tareas más avanzadas.

3. La interrupción de Deepseek en realidad puede beneficiarse de OpenAi. Al demostrar que los modelos de IA pueden ser entrenados de manera más eficiente, Deepseek ha aumentado la demanda general del mercado de IA avanzada. Los inversores creen que Openai está mejor posicionado para capitalizar esta demanda, aprovechando tanto sus modelos existentes como sus iteraciones futuras desarrolladas con métodos más eficientes inspirados en el enfoque de Deepseek.

La revolución de IA que OpenAi encendió hace dos años y medio ahora se está acelerando aún más, y los inversores están convencidos de que la compañía permanecerá a la vanguardia. Su voto de confianza de $ 40 mil millones lo dice todo.

Continue Reading

Noticias

El plan de Google para robar la cuota de mercado de Chatgpt se trata de nivel gratuito de Gemini

Published

on

No es ningún secreto que OpenAi salió, bueno, a todos cuando lanzó Chatgpt hace más de dos años. Ahora, a medida que compañías como Google y Apple intentan competir, están comenzando detrás de Openai, al menos a los ojos del público. ¿Qué tan grande tiene la ventaja de la cabeza? A partir de marzo de 2025, ChatGPT tiene 700 millones de usuarios activos mensuales, lo cual es un total asombroso.

Eso no significa que Google sea impotente en la lucha contra OpenAI para la mayor parte de la participación de mercado de IA de grado de consumo. En los años transcurridos desde que se lanzó ChatGPT, Google aumentó el desarrollo de IA primero a través de Bard, más tarde renombrado Gemini. Mientras que el plan de Openai para monetizar el chatgpt implicó hacer un montón de características tempranas gratuitas y bloquear las avanzadas detrás de un muro de pago, Google está haciendo lo contrario.

Continue Reading

Noticias

Di de Chatgpt y Gemini 5 Image Solts, aquí está quién ganó

Published

on

Tanto Chatgpt como Gemini se consideran entre los mejores generadores de imágenes de IA. Con ambos modelos, puedes describir una escena y verla cobrar vida en segundos. Pero no todos los generadores de imágenes son creados iguales.

Para ver cómo se comparan, le di a ambas plataformas exactamente las mismas cinco indicaciones, desde la fantasía hasta el fotorrealismo, y evalué cómo cada IA ​​respondió en términos de creatividad, detalle y qué tan bien interpretaron el informe.

Continue Reading

Trending