AI chatbots are like that one uncle you see at Thanksgiving. Over his life he became a true expert in something, say, ice sculptures, and can speak with authority on it. But he's convinced he should speak with authority on any subject at all — medicine, law, business, how rainy it will be next spring — stating with 100% certainty things he heard or made up.
Many things he says are right, many are wrong, and you can't judge which is which from his words alone.
My voluminous and authoritative childhood drawings beg to differ
“The Theory That Volcanoes Killed the Dinosaurs Is Officially Extinct” https://m.slashdot.org/story/436739