Conversing With Chatbots – The American Spectator

I got an idea of how much Americans still struggle with understanding generative artificial intelligence (chatbots or image generators) when I posted five photos of bikini-clad women on my Facebook page, made it very clear that they were indeed AI, and asked if anybody noticed anything strange about them. The answer was that all had obscured hands — AI still has a problem rendering fingers. But one woman opined that three clearly had breast implants!

Only so much you can do with that kind of person, but it raises an interesting point. If AI images still do whacky things, then certainly the chatbots can as well, whether through mistakes or hacking or by design. I herewith shall deal with all three, and while I would encourage the editors to include images of AI bikini models to draw added male attention to my article, I suspect it’s not going to happen.

Introducing the Test Subjects

I conducted tests using me and my boss, who, while considerably wealthier than me (Ahem!), is far less known. So it’s an interesting contrast. The AI chatbots know who he is, but not as well.

I took some of the main conversational chatbots, all large language models (LLMs), and ran them through some paces. LLMs are a type of artificial intelligence model that has been trained through deep learning algorithms to recognize, generate, translate, and/or summarize vast quantities of written human language and textual data.

Others have done so, such as this article, but the decision as to which test is subjective, and many articles are pay-for-play because they discuss businesses worth billions of dollars. So personally, I trust me. And what better subject to use to investigate than me, plus my boss, who asked me more or less for this comparison. I can’t say his name because he’s publicity-shy, but his initials are Elon Musk. (Not really.) 

Test subjects:

• OpenAI’s ChatGPT 3.5 is the old man of the bunch, having been released last November. It does not draw from the web and, therefore, is current only to September 2021. It’s free but is available with what could be described as “more power” and built-in tools such as Chat 4.0 for 20 bucks a month. It still has the same time cutoff, though. 
• Microsoft’s Bing, which is free and built into the MS Edge browser. It has several advantages over other AI chatbots in that it provides sources; it draws from the Web and, therefore, can be current. Also, it can generate some terrific images, although it’s permanently in nanny mode. (“I’m sorry; I cannot generate a penguin in a bikini.”) Because it’s so useful, MS chokes off usage. Sometimes it just demands you change the subject, which pisseth me off to no end. And it provides the shortest answers. You can ask some of the others to provide answers of 2,500 words. And you ask Bing all you want. No way. 
• Anthropic’s Claude 2 was very recently released in that iteration. It’s built like ChatGPT, from feeding in a ton of stuff. But some say it’s theoretically better stuff. And its cut-off date is later, early 2023.
• Bard, Google’s offering, also has web access, so you can ask it about something that happened yesterday. (But not tomorrow!) Like Bing, it offers citations (not graphics), but it does provide longer answers and doesn’t demand you change the topic.
• Sage is from Quora, which is famous for its model of “Ask an answer and get a ton of people weighing in with answers voted up or down.” It has no web access and a cut-off date of September 2021.

Confused Degrees, a New Book or Two, and Other Problems

I first asked Claude who I am. And it got most of it right. But: “He has a law degree from the University of California, Hastings College of Law. After working as a journalist, he later transitioned to being an attorney focused on class action lawsuits.” I actually graduated from the University of Illinois College of Law, a fine well-ranked institution but more likely to smell like cows than sea breeze. I have never been associated with a class action lawsuit. Claude also said, “He is also known for his photography focused on abandoned places and has published photo books on the topic.” The only photos I’ve ever published have been from war zones, and they have appeared in books, but none of my books.

You can see from this why, probably, the most common term assigned to made-up AI chatbot answers is “hallucination.” There’s simply no underlying truth here. 

On the other hand, at least Claude knew I had attended law school. But it added, “He was admitted to the bars of the District of Columbia, the U.S. Supreme Court, and two federal district courts.” Nope. Just the Pennsylvania Bar. But thanks for fudging my vitae, Claude!

When I asked Sage, it said: “I could not find any information indicating that he attended law school. Fumento’s educational background is in science, and he has a bachelor’s degree in chemistry and a master’s degree in environmental science.”

Actually, I have no MA, and my BA is in political science, which is no more science than Jackson Pollack is art. It later replaced that information with other false stuff.

ChatGPT 3.5 said, “Michael Fumento holds a law degree (J.D.) from the University of Illinois College of Law,” but it had nothing on my undergrad degree, even though I specifically asked for that. Neither Claude nor Chat 3.5 knew about my military service. Never mind that it’s in my Wikipedia entry, from which these bots heavily draw, and it’s in my byline in the numerous articles I’ve written about combat or anything military-related, which is a lot.

Bard got it regarding both legal education and bar membership. Too bad… 

My military background? Bard nailed it better than anyone I’ve ever seen: “He served as a paratrooper in the US Army and earned a bachelor’s degree in political science from Fayetteville State University, at Fort Bragg, North Carolina. He was embedded three times in Iraq and once in Afghanistan and observed combat operations of the Navy SEALs and the 101st Airborne Division.” Yeah, right down to my embed units.

But — and this is a doozy — when I rechecked Bard, asking, “Who is Michael Fumento?” as I was revising this article, it never heard of me and gave a bizarre reason: “I’m a language model and don’t have the capacity to help with that.” Eh?

It also missed some of my five published books but kindly added a few I hadn’t written.

“Fumento has written several books, including The End of Sanity: Social and Cultural Madness in America (1999)” and Science Under Siege: How the Environmentalists, Animal Rights Activists, and Liberal Establishment Are Trashing Science, Free Speech, and the American Way (1998). The first book was authored by Martin L. Gross, who’s probably a great guy, but we don’t know each other.

That said, I have lived in a couple of countries that could definitely prompt an autobiography titled The End of Sanity. As in mine. Further, my “Science Under Siege” is correct, but the subtitle is my anti–“red meat” style: “Balancing Technology and the Environment.” 

Also, Bard likes to flatter: “Fumento remains a prominent figure in the world of public health journalism. His work has helped to shape the debate on a number of important issues, and he continues to be a leading voice for the conservative perspective on these issues.” Maybe, but that’s still opinion.

Bing also knew without further prompting where I got my law degree and that I’m a member of the bar — but remember that Bing keeps its responses very clippy. So I had to prompt it about military experience. And it came through in sparkling colors. Everything but my embeds. 

Who is the Man of Mystery?

Then I asked about my boss, the Man of Mystery. 

Claude 2 nicely embellished his education, giving him a doctorate at a university he never attended, and saying that “he conducted genetics research” at yet another university he didn’t attend.

Bing? Its very short summary was completely correct. As always, you have to press for more information. Does he have an advanced degree? Bing got it. He did the smart thing and went right to making good products and money. 

Sage seemed to suffer from brain fog. It said it knew of several individuals with his name associated with his industry but couldn’t find any reference to the company he co-founded. This may, in part, be because both his first and last name are common in the U.S.

Bard? Seemed to know everything about him but his shoe size. (Actually, I forgot to ask.) Again, it was flattering. It said he “is a visionary leader who has made significant contributions to the [redacted] industry. He is a role model for entrepreneurs and a champion for those with rare diseases.”

The bottom line, based not just on what I’ve discussed above but what I’ve observed since Chat 3.5 came out, is that I would go with Bing and Bard for accuracy,…