If you've got a concern about errors and inaccuracies in AI output, how do you address it?
To my knowledge, there is not yet a solution for AI hallucinations.
I recently wrote a newsletter on LinkedIn and Substack about my concerns about inaccuracies and pure fantasy I discovered using Bing powered by ChatGPT-4. This is a verbal analysis of my thought process of why I wanted to pursue this question.
I'm including the text of the newsletter for reference.
The Seeds of Curiosity
One of my first podcast episodes was about AI use in HR - recruiting and retention, upskilling etc. Although Todd Raphael changed jobs, he allowed me to retain the episode (thank you, Todd). It was really useful because it made me curious to learn more.
I started investigating Canadian companies doing the same sort of work and I discovered Plum.IO. I created a profile and though it wasn't bad, there were several things I thought were inaccurate. You can read my Plum profile here.
To be clear, I like a lot of it. My top talents are embracing diversity, innovation and decision making. It describes me as Influential, dominant, persuasive, authoritative, and self-confident. I'm also curious, analytical, reflective, imaginative, and creative as well as receptive, adaptive, flexible, unconventional, and reflective.
I agree with these statements:
Nola is:
Good at knowing what to say to make people feel included and accepted.
Exceptional at adapting to a wide range of people and being open to different values and personalities.
Nola is:
Extraordinary at working with abstract ideas and developing unconventional solutions.
Exceptional at being receptive to different ideas and flexible to changing demands.
Nola is:
Extraordinary at critically evaluating information, generating solutions, and using facts and logic to support decisions.
Excellent at expressing opinions and taking control of a situation.
I have an issue with several of these statements:
Professionally, Nola is:
A good fit for roles with clear and attainable goals.
A good fit for roles that are unlikely to have many unexpected problems or changes in how the work gets done.
A very good fit for work that is not stressful or rushed.
A very good fit for roles that measure success on the volume of work completed.
The hybrid/remote and the future of work is not clear, it's likely to have many unexpected problems and changes, it can be stressful and how do you measure the volume of work completed? This is not describing me or the work I do at all.
To me, this was picking up my long history in customer service and attributing the ideal work to my history. I was curious how the AI was determining this output and so I asked the CEO to be on my podcast. I had questions.
How does the AI pull information and create the output?
How do you fix output you feel is inaccurate? What recourse is there?
How would employers view this and how would a candidate counter misinformation provided by AI in an interview situation?
What protections do you have in place to guard against bias?
If I could see this many problematic issues just on my profile, how would this look for others? What happens if you don't have 2 decades of experience? What if you are an immigrant?
The Plum CEO initially responded but has since ghosted me. We were actually on the same stage at Elevate Festival last September - I did send her a message asking to meet. She walked right by me - she presented with a VP of Scotiabank right before my speaking gig. Scotiabank doesn't use resumes anymore - they use Plum. This is not a future of work issue. This is a now issue. It's real, it's happening already.
Although I'm happy to chat with the CEO if she approaches me, I'm not going to pursue it further. This is the story I have. The questions remain. AI is a content track for Elevate Festival this year again, as is the future of work. Hopefully Jared Lindzon or Douglas Soltys has an opportunity to pose some of these questions to the AI experts in attendance.
Who Can It Be Now?
AI has a tendency to hallucinate. I recently discovered you can enable ChatGPT-4 when you use Bing. I like Bing because it uses the internet.
A few weeks ago, I interviewed Cher Jones about personal branding. Just before our call started, I searched her on ChatGPT-3. It was a decent summary. I looked for myself and it told me it knew nothing about me but the data was only accurate until September 2021. As far as it was concerned, I was a private person.
Which made me wonder where it gets it's information. I've been on most social media platforms since 2011. It's clearly not valuing social media as a source. From the research I've done over the last few weeks, it seems to value books, websites and public media.
How do you build a personal brand when you can't see inside the box?
Who Are You? Who, Who?
AI has a tendency to hallucinate. I recently discovered you can enable ChatGPT4 when you use Bing. I like Bing because it uses the internet. Yesterday I asked: tell me about Nola Simon’s media appearances. Can you spot the things that aren’t true or completely accurate? Two people I know fairly well played along. One told me that she felt the list was pretty impressive and credible and the other person was able to spot most of the inaccuracies. Another person on Threads told me that she would have no way of knowing what was true and what wasn't. No surprises the people on LinkedIn know more about me than the people on Threads.
I've quoted what Bing wrote below. Can you spot the things that aren’t true or completely accurate?
Probably you can't without doing deep research. Due diligence would demand you verify each statement. How time consuming. I'll break it down for you.
Sweet, Sweet Fantasy, Baby
Nola Simon has been featured in various media outlets, such as:- Newspapers: She has written articles for The Globe and Mail, The Toronto Star, The National Post, and The Financial Post on topics such as hybrid work, employee wellness, and leadership¹².- Television: She has appeared on CTV News, CBC News, Global News, and BNN Bloomberg as an expert on hybrid and remote work, and the future of work¹².- Ra. Itdio: She has been interviewed on CBC Radio, AM640, and SiriusXM Canada Talks on how to navigate the changing world of work, and how to have difficult conversations with employers and employees¹².- Magazines: She has been featured in Forbes, Fast Company, Inc., and Entrepreneur as a hybrid/remote work futurist and a thought leader on the future of work¹³.- Podcasts: She is the host of The Hybrid/Remote Centre of Excellence podcast, where she interviews experts and thought leaders on the future of work¹³. She has also been a guest on several podcasts, such as The Change Zone, Work Life Bloom, and The Janus Oasis².
Let's take it section by section.
All of the real interviews can be found on my media page on my website.
Newspapers
- Newspapers: She has written articles for The Globe and Mail, The Toronto Star, The National Post, and The Financial Post on topics such as hybrid work, employee wellness, and leadership.
I didn't write any of the articles mentioned. I was interviewed by talented journalists like Rosa Saba Clarrie Feinstein and Déjà Leonard. It misses an article by CBC Geoff Nixon. It doesn't mention The Toronto Star at all. I've never been interviewed by The National Post or The Financial Post although I realize some articles are distributed. The recent article Rosa Saba wrote for The Canadian Press was published by 37 different outlets and audio clips also ran across Canadian radio.
These inaccuracies in the AI output only highlight why we need human journalists who do deep research and investigation. We also need the editors and producers who do the background work, planning and strategy. I have learned a lot about media since last year and I have nothing but respect for the work.
We need brave, human journalists who invest the time and effort to tell real stories.
- Television: She has appeared on CTV News, CBC News, Global News, and BNN Bloomberg as an expert on hybrid and remote work, and the future of work¹²
Television
I have appeared on CTV News - TV was actually my first media interview thanks to Beth Macdonell. I have not appeared on television for CBC News, Global News or BNN Bloomberg. If they care to have me, I'd love the opportunity but this is fake news, AI-style.
Radio
- Radio: She has been interviewed on CBC Radio, AM640, and SiriusXM Canada Talks on how to navigate the changing world of work, and how to have difficult conversations with employers and employees.
So radio is the most complex media. I had no idea until I started doing interviews for CBC Radio. It's hard to know when the shows will air - the only time I knew for sure was the day I did a series of live interviews across Canada. CBC Spark with Nora Young goes live on Sundays at 1 pm but also publishes as a podcast. They do have a channel on SiriusXM but I have not been interviewed for a show called SiriusXM Canada Talks and I've not been on AM640 to my knowledge. As I mentioned, Canadian Press distributed audio clips from my recent interview but it's almost impossible to know which stations broadcast those.
Magazines
- Magazines: She has been featured in Forbes, Fast Company, Inc., and Entrepreneur as a hybrid/remote work futurist and a thought leader on the future of work.
Pure fiction.
The closest I came to being published in a magazine is the interview with Wendy Helfenbaum for a digital publication called Reworked but is that considered a magazine or a newspaper? Not sure. I would call it digital media but there is no category for that and Bing didn't pick this up at all. It also didn't pick up any of the quotes I gave Terkel (now Featured).
Podcasts
Although I have a Spotify playlist with all my guest podcast interviews, Bing didn't use that as a resource at all.
- Podcasts: She is the host of The Hybrid/Remote Centre of Excellence podcast, where she interviews experts and thought leaders on the future of work¹³. She has also been a guest on several podcasts, such as The Change Zone, Work Life Bloom, and The Janus Oasis.
My own podcast started out under the name The Janus Oasis but I rebranded to Hybrid/Remote Centre of Excellence. I've done several episodes for The Change Zone with Gail McDonald - PCC and Susan Sneath so I'm happy to see that show listed but the most interesting fallacy is the Work Life Bloom show.
It doesn't exist.
Dan Pontefract is publishing a new book called Work Life Bloom which comes out in October in Canada and November for the rest of the world. He has been a guest on my podcast but I have not been a guest on his podcast. His podcast is called Leadership Now. This is the most creative AI hallucination. And the most worrisome. It takes a kernel of truth but builds out a fantasy. It seems plausible. Maybe you've seen some of Dan's advertising for the book. It's credible he might have a podcast of the same name. Dan is also a main stage speaker at Elevate Festival this Wednesday. Perhaps you can ask people how this output would have been created, Dan?
Call To Action
We need more information about how AI is pulling information. The hallucinations have the potential to impact personal branding and reputation. We need to ask questions and we need to understand how we fix the hallucinations. If we can't fix them, we need to know how to clarify the facts. We need to know how to negotiate disinformation.
How can we possibly use this for HR when there is no answer on what is causing the hallucinations or how to fix them?
In short, we need to know who and what to trust.
AI Hallucinations: Influencing the Conversation