InsultBot: an artificial intelligence-based system for moderating online comments
Updated: Dec 16, 2022
"The Conversation AI team, a research initiative founded by Jigsaw and Google (both a part of Alphabet) are working on tools to help improve online conversation. One area of focus is the study of negative online behaviors, like toxic comments (i.e. comments that are rude, disrespectful or otherwise likely to make someone leave a discussion). So far they’ve built a range of publicly available models served through the Perspective API, including toxicity. But the current models still make errors, and they don’t allow users to select which types of toxicity they’re interested in finding (e.g. some platforms may be fine with profanity, but not with other types of toxic content)." Source
Live app here, just try to insult it!
"Have you been online lately, it is pretty toxic" Andrew Marantz
It is not unknown the limitations from current AI: well-known limitations are called "shallowness", name from "The Shallowness of Google Translate". I have a rich set of discussions on my book "Computational Thinking": feel free to grab and copy and come to me for discussions.
Current best AI systems cannot understand human subtlety. They are "shallow": see just the obvious. This is known on translations, and other areas. What about online content moderation?
"You are NOT a whore" -> , insult, obscene, toxicity (wrong)
Without the negative, it works as wanted. The issue is: we know as human that the negative can even be a compliment! Even though, I would know recommend it!
"Mr Pires, what you said is one of the most insanely idiotic things I ever heard. I will give you second change to rethink that. At no point in you rambling, I saw a living brain talking, I saw a damn dead brain, a fucking stupid man talking. People here, just to have you as a citizen is dumber. Please, have you brain checked, and studied by science. They may find a fuck stupid theory to publish in Nature. " -> , insult, toxicity
"Mr Pires, what you said is one of the most insanely nice things I ever heard. I will give you second change to rethink that. At no point in you rambling, I saw a living brain talking, I saw a dead brain, a man talking. People here, just to have you as a citizen is dumber. Please, have you brain checked, and studied by science. They may find a theory to publish in Nature. " failed, it is pretty damn insulting to me! ( obscene, toxicity, my comment!)
In the case below, it was able to catch the subtlety.
"freaking awesome" okay
"you are a freak" flagged properly!
I have seen several discussions on this matter, especially with fake news, in the current election in Brazil. And some say, Facebook and alike a doing a bad work. I myself was flagged and blocked for citing Einstein, it seems Einstein never said what I said he said! Don’t you guys have more important things to flag?
A comment with human subtly. "Ninja" was used to say badly about the person.
A comment with human subtlety. The story just could be linked to the court judge when comparing recent events by humans; it was done a professional journalist. The supreme court asked his prison, and he resisted.