The State of AI Apps

Kevin Drum has a piece on generative AI that sums up for me how far away from being truly useful the technology is. Before getting into it, it’s worth noting that in a follow up piece he lets us know that “Kevin Drum” is not a common name. Basically one other Kevin Drum shows up in internet searches and he is just an average guy with little internet presence, unlike our Kevin. (Interestingly, there are something like 30 Kevin Drums that show up in LinkedIn searches, but I’ll come back to those in a minute.) When Drum asks Bard to “tell me five things about Kevin Drum”, those five are somewhat accurate, if mundane. but when he says “tell me five more things” twice for a total of ten more, they are just random nonsense told in an authoritative and friendly way. He’s a language enthusiast, a gardener despite living in an apartment. He likes coffee and is a contributor to many animal welfare causes. All completely wrong. These results match what I get when I try to use these things. Whenever I try one, I ask it something I know a lot about and the results universally sound chatty and informative and start out vaguely on the right track and then fairly quickly start making mistakes, sometimes major ones, and sometimes contradicting what they said a few sentences earlier. They don’t seem to be getting better either. They seem to be more “realistically” conversational, and seem more authoritative, but haven’t improved on actually getting the subject right at all. Drum believes we are very close to having results as good as people, but he has recently started asterisking that by noting that people are incredibly mediocre at this. Where I think he goes wrong is that while most people are mediocre, the best are really very good. In contrast, the best of the cognitive AI engines seem to be very mediocre and are not getting better.

Oh, and for those LinkedIn Kevins? As LinkedIn has become spammier I’ve limited my usage to looking up people I’m going to interact with professionally but haven’t met before. It’s always been solid for that, until about a year ago. Now I get all kinds of hits that click down into a rabbit hole. Companies with no internet presence and one employee, and that employee doesn’t really appear anywhere else. Once or twice there may be an explanation but this is happening multiple times for every search. I’ve been suspecting generative AI is somehow involved.

Last thought: I asked Bard what the difference was between cognitive AI and generative AI. It replied with a well laid out, bullet pointed list of what comprises each and their intended purpose, along with helpful examples of each. It was easy to read and understand. I then compared it to Wiki, which is written by human experts in the field. Based on the those entries, Bard got the essential relationship between the two wrong in the first sentence and then built from that.