Follow
Interesting slides and talk notes from Arvind Narayanan on How to Recognize AI Snake Oil https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf
In a nutshell: There are things AI is ok at or getting better, but predicting social outcomes isn't one of them.
@whami None of this surprises me, unfortunately. A damn shame, though, that they used that single day out of context. At least you now how proof they'll behave that way.
@auntiekiki There is a function in my work involving making predictions of crowd dynamics that my team is reasonably good, but not great, at performing. A while back the Advanced Analytics team took a crack at it. We were told that their bot ended up with 88% accuracy, which is astounding. There was a huge push to start transferring over to this. Before the bot went live someone succumbed to the pressure of questioning from a peer and me, and admitted that the bot had a single day where it was 88%, but on every other day of the trial it was between 17% and 44%.