AI Is the Latest Talking Platypus

Cartoonish picture of a platypus standing in a field making a hand gesture and talking

I was talking about AI last week with Yochi Cohen-Charash and how it often disappointed. We decided to try an experiment. We asked Copilot to create a biosketch for each of us. What we discovered is that it wrote the same sketch, just substituting details. She was a leading researcher who studies emotion. I was a leading researcher who studies stress. Copilot was just taking a boilerplate and changing a few words here and there. We tried a few other academics and then some practitioners. Everyone had more or less the same bio, with the same phrasing and superlatives, for example,”Throughout his career, ABC has made significant contributions to the field of XYZ”. It suddenly hit me, AI is the latest talking platypus.

The Talking Platypus

The parable of the talking platypus is the simple phrase, “if a platypus can talk, does anyone pay attention to what it says?” I first came across it at a conference in the late 1970s where someone used it in the title of a paper. The study they reported had to do with perceptions of women in engineering, a rare phenomenon at the time. They conducted an experiment where engineering faculty were asked to evaluate the quality of a research paper, randomly assigned to appear written by a man or a woman. When attributed to a woman, the rating was higher than when attributed to a man. The explanation was that there were so few women writing engineering papers in the 1970s, that people were surprised to see one, and that surprise affected their ratings.

AI Is the Latest Talking Platypus

AI is the perfect talking platypus. What it can do is amazing and impressive. You ask it to write an essay, and it produces something in perfect English prose. I find that every time it gives me a written product, at first read it is beyond belief. A computer program produced something with incredible flow and polish. Then the excitement wears off and I read it a again, this time more critically. I see the flaws. The writing is formulaic. For a bio sketch, the superlatives seem like it was written by a marketing promotion company to put the person in the best light possible. Then I notice the inaccuracies–the hallucinations. My sketches have had me doing work on topics I don’t do, being editor of a nonexistent journal, and being president of a society I am not even a member of. Somehow the AI has pulled together sources that provided facts, and filled in the gaps with things that sound great, but aren’t true.

It Is Time To Get Past the Hype

AI is a potentially valuable tool, but it will take time for us to figure out how to make the best of it. I have used it with mixed success to do a variety of tasks. Think of AI as an assistant that needs close and constant supervision. You can give it a task, but you need to pay attention to the details and not get swept up in the talking platypus phenomenon.

Some of the things I’ve used it for include.

  • Writing discussion cases. I gave it a scenario, “write me a case about a server giving excellent customer service”. This it did extremely well. It is a piece of fiction so hallucinations are not a concern. I grade it an A+.
  • Doing content analysis and noting themes for a qualitative study. Feed it written material and ask it to identify themes and what it produces is not very different from human coding. Sifting through material and looking for common elements and producing a summary is what AI is all about. I give it an A.
  • Doing content analysis and noting representative quotes for each theme. This was a total fail. Most of the quotes were hallucinations. When I checked the quote against the original material, most didn’t exist. They all sounded great and reflected the sorts of things people said, but they were hallucinations. Grade: F.
  • Making images. I’ve been using DALL-E 4.0 to make images for my blog posts. What it produces is good enough for my purposes, but I have a difficult time getting it to give me what I want. I ask for 4 people sitting around a table and it gives me 6. I ask it to eliminate 2 of them and the next iteration now has 5, but it tells me there are 4. DALL-E can’t spell either. Ask for a sign that says something, and most of the time you don’t get it. Even worse, when I asked for a picture of a nurse being yelled at for an article on mistreatment of nurses, I got a lecture about how I shouldn’t be asking for such a negative image. Grade: B-.
  • Finding sources. I was looking for a source that made a particular point, so I asked Chat-GPT to provide one. It gave me the references for three sources. They were all on the topic, but none of them made the point. I grade no higher than a C.

My conclusion after having used AI for a while is that it is better at fiction than nonfiction. Ask it to generate fictitious examples, and it will shine. It can be really hard to think up a hypothetical, but AI can do it quickly. You just have to do a little editing. When it comes to research, you have to be very careful of hallucinations. Ask it to write a summary of a topic, and likely it will not be entirely accurate. You need to fact check everything. I also find that when I ask it to write on a topic I know about, that I do not agree on what it chose to include and not include. This shouldn’t be a surprise as AI is not an expert in a field. It has access to a tremendous amount of content, but what it chooses to include is based on something other than expertise.

AI is the latest talking platypus. Over time there has been a tempering of views as more people have tried it and became familiar with the limitations, so moving forward perhaps the platypus effect will fade. AI is definitely a valuable tool, and certainly it will continue to see increased use moving forward. But we all need to be wary of the talking platypus effect and keep in mind that AI needs close human supervision to maintain quality control on its products.

Image generated by DALL-E 4.0. Prompt: Image of a talking platypus. Less cartoon style. Make the image wider.

SUBSCRIBE TO PAUL’S BLOG: Enter your e-mail and click SUBSCRIBE

Join 1,233 other subscribers

1 Reply to “AI Is the Latest Talking Platypus”

  1. Dr. Spector, I appreciate this article. Yes, AI can be useful and requires diligent oversight. This article is a helpful reminder to critically evaluate any tool used as a means of simplifying or easing efforts in completing tasks.

Leave a Reply

Your email address will not be published. Required fields are marked *


The reCAPTCHA verification period has expired. Please reload the page.