A joint study carried out by researchers from Alphabet's Jigsaw and the Wikimedia Foundation has analyzed all user comments left on Wikipedia in 2015 in order to identify how and why users launch in personal attacks, one of the many faces of online abuse.
To analyze the gigantic trove of sample comments, researchers developed a machine learning algorithm that was able to identify and distinguish different forms of online abuse and personal attacks.
In order for the algorithm to work, it had to be trained beforehand. For this, researchers used human users to classify a small batch of 100,000 comments, with each of the test comments passing through the hands of ten different humans.
The resulted data classification allowed the algorithm to accurately distinguish between direct personal attacks (statements like "You suck!"), third-party personal attacks (statements like "Bob sucks!"), and indirect personal attacks (statements like "Henry said Bob sucks").
After training the algorithm and unleashing it on all Wikipedia 2015 user comments, researchers were able to identify personal attacks, and then collect data on the users that launched them.
Their findings reveal that around 43% of all comments left on Wikipedia came from anonymous users, but most of these were one-time commenters, and the number of comments they left was 20 times smaller than comments left by registered users.
Despite this, researchers discovered that anonymous users were six times more active in posting personal attacks, but in the end, they contributed to less than half of personal attacks, meaning a large number of personal attacks came from users with a registered identity on the site.
Of all personal attacks, researchers noted that about a tenth came from extremely active users, who had an activity level of 20+, the highest on the site.
A closer look at the data revealed that 34 "highly toxic users" from this 20+ category were responsible for almost 9% of all personal attacks on the site.
"By comparing these figures, we see that almost 80% of attacks come from the over 9000 users who have made fewer than 5 attacking comments," the research team noted, something that's somewhat normal, as everybody tends to get mad at one point or another.
"However, the 34 users with a toxicity level of more than 20 are responsible for almost 9% of attacks," researchers noted.
"Thus, while the majority of Wikipedia’s attacks are diffused infrequent attackers, significant progress could be made by moderating a relatively small number of frequent attackers," researchers concluded.
This suggestion from the research team comes as the efforts from Wikipedia's moderators have been sub-par, which was also one of the facets of their research.
"Our analysis takes all attacking comments in the 2015 Wikipedia corpus and asks how many of these lead to a moderation event in the following 7 days. We find that 7.7% of attacks are followed by a warning and 7.0% of attacks are followed by a block within this period."
Furthermore, the research team also focused on patterns that lead to personal attacks. One of their main findings was that personal attacks were clustered together in a small number of threads.
This led researchers to conclude that one personal attack led to another, and another, and another. Users tended to think that if one user got away with abusive behavior, they could do the same, which is why Wikipedia moderators could act more decisively and shut down abusers right away before comments get out of hand.
The research team hopes that Wikipedia uses their study to build a comments monitoring dashboard that could track down hotspots of abusive personal attacks and help moderators ban or block toxic users.
More details are available in the research paper titled "Ex Machina: Personal Attacks Seen at Scale."