What you need to know
- Microsoft recently announced a new Bing powered by ChatGPT.
- It has since been discovered that Microsoft’s demo of the new Bing included several factual errors.
- The search engine shipped to a wave of testers earlier this week and has generated many responses with inaccurate information.
- In one instance, Bing shared a list of ethnic slurs.
Microsoft unveiled a new Bing with ChatGPT during an event last week. The search engine made headlines across a range of sites, from tech blogs to general news sites. But it has since been discovered that Microsoft’s new Bing made several factual errors during a demonstration that went unnoticed.
Dmitri Brereton shared a blog post highlighting several mistakes made by Bing. The search engine showed factual errors in several categories, but perhaps the most significant mistakes were financial errors. The tool confused financial terms, such as gross margins and adjusted gross margins.
Mistakes like those can escalate quickly. Bing’s failure to differentiate between different categories can create comparisons that are entirely inaccurate. Another factor is that AI, including the new Bing with ChatGPT, often exudes confidence when sharing incorrect information. If you asked a person with limited financial knowledge to make a similar comparison, they might admit that they are not clear on the difference between gross margin and adjusted gross margin. Bing did not do so in this case.
At one point during the demonstration, Bing shared figures that didn’t appear in its source material at all. Brereton dives into detail, but the long and short is that Bing appeared to have completely fabricated financial data when creating a summary of a document.
It’s not just financial data that Bing got incorrect during its demonstration. Brereton pointed to the search engine incorrectly stating a vacuum had a cord. That’s yet another example of an inaccuracy that would render a comparison between items useless.
Not just the demo
Now that Bing is rolling out to preview testers, people are running into other mistakes as well. A Reddit post shared an example of Bing claiming it was currently 2022 and that a person would have to wait 10 months to see a film that came out last year. Bing even suggested that the user may have a virus or bug on their phone that caused an incorrect date to appear.
Microsoft responded to that particular issue and shared a statement with The Verge, “We’re expecting that the system may make mistakes during this preview period, and the feedback is critical to help identify where things aren’t working well so we can learn and help the models get better.”
Bing has also taught people ethnic slurs, as reported by PCWorld. The search engine did add a note that “nicknames for various ethnicities can be hurtful and disrespectful” when pushed to share more, but it had already listed several offensive terms at that point.
Note: The image below includes slurs and derogatory terms for several groups and ethnicities that were generated by Bing.
Microsoft discussed the situation shown above in a statement to The Verge:
“We have put guardrails in place to prevent the promotion of harmful or discriminatory content in accordance to our AI principles. We are currently looking at additional improvements we can make as we continue to learn from the early phases of our launch. We are committed to improving the quality of this experience over time and to making it a helpful and inclusive tool for everyone.”
Google gaff vs Bing blunder
Google’s Bard AI drew criticism when it was discovered that the tool included factual errors within an advertisement. Alphabet shares dropped 7%, which equated to over $100 billion in market cap.
It’s unclear if mistakes generated by the new Bing and ChatGPT will result in similar losses for OpenAI and Microsoft. There are some differences between the situations. First, Google is the established leader in search. People are going to be less forgiving with anything Google rolls out than OpenAI’s ChatGPT and Microsoft’s Bing.
Additionally, bad news gets more traction than good news, so people are more likely to jump on the dominant search platform making mistakes than an up-and-comer having issues.
Windows Central take
In my own testing, I’ve had mixed results as well. The very first question I asked the new Bing after gaining access yielded an incorrect response. In fact, the response wasn’t just wrong, it contradicted itself. I asked Bing about how Patrick Mahomes did in the Super Bowl.
The response stated that Mahomes had won one Super Bowl and then later claimed he had won two. The latter is accurate but only became accurate as of Sunday at the conclusion of Super Bowl XLVII. But a search engine needs to use up-to-date information to yield results. The example above also shows that Bing does not fact-check against itself, at least to the level it needs to.
When we covered Bing going off the deep end, our Editor-in-Chief Daniel Rubino highlighted that AI gets better exponentially. He’s right. Bing and ChatGPT will get better with time as they are fed more data. But Microsoft still needs to be careful.
Interest in Bing is at an all-time high. Google’s Bard rightfully received criticism when it shared an error in an ad. Every Bing blunder puts the search engine at risk of losing momentum.
Microsoft shouldn’t be immune to criticism just because its technology is exciting. Yes, the new Bing is in preview. It will make mistakes and improve over time, but Google took its lumps when Bard made a mistake. Now it’s Bing’s turn.
