De-platforming Facebook COVID conspiracy theorists has not significantly reduced their influence, says research

Removing high-profile COVID conspiracy theorists from Facebook has had only limited impact upon the spread of misleading information, research from Cardiff University has found.

Fan pages, affiliated groups and other secondary accounts set up by devoted believers continue to share problematic content about the causes and consequences of COVID-19 long after primary accounts have been taken down, the findings reveal.

The team, from Cardiff University's Crime and Security Research Institute (CSRI), say these "minion accounts" also increase the resilience of conspiracists by encouraging them to diversify their presence across an alternative network of platforms, personal websites and subscription services.

Published in the peer-reviewed journal Information, Communication and Society, the study examines Facebook's de-platforming of David Icke and Kate Shemirani for repeatedly violating its policies on harmful disinformation during the pandemic, and how followers of these individuals responded to the interventions to limit their overall effectiveness.

Lead author Dr. Helen Innes, Research Fellow at the CSRI, said, "Our research shows how de-platforming can sometimes reinforce support for charismatic conspiracists like Icke and Shemirani amongst their adherents. Their removal from Facebook, Twitter and YouTube is framed as a badge of honor, evidence that their actions worried mainstream big tech enough to act against them.

"When this happens, our data shows how their followers mobilize by setting up new fan pages, public and private groups, sharing links and videos, and setting social media challenges in order to continue a mission which they feel is being censored.

"In this context, de-platforming has limited success in actually disrupting and suppressing the harmful behavior. Instead, for many it actually reinforces their conspiratorial beliefs, as what we term 'minion accounts' double down on spreading the disinforming material on behalf of those who have been de-platformed."

In the four weeks following the de-platforming of Icke, a well-established conspiracy theorist, researchers analyzed 11,877 public posts that mentioned him. These posts yielded 2.2 million user interactions. In the seven days following his account removal on 30 April, his public mentions on Facebook increased by 84%.

The team analyzed 1,636 post mentions of Kate Shemirani during 2020. Her profile grew in prominence during the pandemic before she was de-platformed on 4 September 2020.

In the two months following her removal, post mentions and user engagement on Facebook decreased markedly. However, the suppression effect appeared to be temporary, with signs of revival from the end of 2020, where the number of Facebook video shares increased from approximately ten in October and November 2020 to over 60 in the next two months.

Facebook's measures to stop misleading information were ramped up in 2020 in response to the pandemic and prior to the US election. De-platforming, which is Facebook's ultimate sanction, was applied to accounts assessed as posing risks to public safety, defined under its Dangerous Individuals and Organizations Policy.

In the year to September 2020, Facebook took down more than 1 million groups for repeat violations, implementing new counter-measures intended to prevent the administrators of those groups from creating new ones.

Co-author Professor Martin Innes, Director of the CSRI, said, "There is little doubt that despite Facebook's efforts to combat the spread of harmful misinformation, more effective action is required from big tech in order to combat the deluge of conspiracy theories that we've seen over the course of the pandemic.

"If, as our study suggests, de-platforming can have complex effects in terms of both constraining the public reach of harmful content, whilst simultaneously inducing greater resilience among these groups, then further measures will need to be introduced to ensure misleading information is not allowed to thrive online."