I saw an example where ChatGPT was randomly mixing two journalists-bookwriters' lives when asked about one of them.
A bit like asking "who's Lionel Messi" and getting back that he's a Portuguese player known for career in Real Madrid. Then you'd ask about Cristiano Ronaldo and it may tell it's an Argentinian World Cup winner who spent his career at FC Barcelona (example invented by me to illustrate the point).
Basically it takes some knowledge from a bucket and adjacent buckets and doesn't really distinguish that the concepts are not the same if they're related close enough.
A bit like asking "who's Lionel Messi" and getting back that he's a Portuguese player known for career in Real Madrid. Then you'd ask about Cristiano Ronaldo and it may tell it's an Argentinian World Cup winner who spent his career at FC Barcelona (example invented by me to illustrate the point).
Basically it takes some knowledge from a bucket and adjacent buckets and doesn't really distinguish that the concepts are not the same if they're related close enough.