Methodology
The investigative team analysed disinformation content about the
Died Suddenly
documentary posted on nine different platforms
18
between October 27, 2022 and December 28, 2022, that met the following criteria:
-
Were fact-checked by reputable fact-checking organisations.
19
-
Were rated âFalseâ, âMisleadingâ, âUnsupportedâ, or âIncorrect.â
-
Could cause public harm by undermining public health. Avaaz has included content that impacts public health in the areas of:
-
Preventing disease:
e.g., false information on diseases, epidemics and pandemics and anti-vaccination misinformation.
-
Creating distrust in health institutions, health organisations, medical practitioners and their recommendations:
e.g., false information implying that clinicians or governments are creating or hiding health risks.
-
Fearmongering:
health-related misinformation that can induce fear and panic, e.g., misinformation stating that the coronavirus is a human-made bio-weapon being used against certain communities or that Chinese products may contain the virus.
METHODOLOGY FOR MEASURING LABELLING AND REMOVALS
For the purpose of measuring the fact-checking efforts of major social media platforms, the Avaaz investigative team analysed a sample of 108 posts about
Died Suddenly
based on the above criteria.
For each of the false and misleading posts and stories sampled based on the above criteria, the team recorded and analysed, using both direct observation and CrowdTangle:
20
-
The total number of interactions it received.
-
The total number of views it received in the case of video.
-
Whether each had a warning label as false or misleading, or whether the given platform took down the piece of content.
-
When misinformation posts would receive a fact-check warning label or be removed.
21
METHODOLOGY FOR CALCULATING THE SPREAD OF THE OFFICIAL DOCUMENTARY LINKS ON FACEBOOK
On December 6, 2022, the Avaaz research team studied the spread of the three following official movie links created by the
Died Suddenly
official accounts on Rumble, Twitter and on the Stew Peters official website.
Using CrowdTangle,
25
Avaaz selected the 180 publicly available posts
26
that had shared one of the above mentioned links and had garnered a minimum of ten interactions.
METHODOLOGY FOR CALCULATING THE SPREAD OF LINKS ABOUT THE DOCUMENTARY ON FACEBOOK AND TWITTER
Science Feedback, a fact checking organisation,
27
provided Avaaz with a dataset of 52 different links
28
29
to the official
Died Suddenly
full movie, trailer or articles reporting on the documentary. This dataset included 38,977 tweets that shared these links.
We selected the 44 links that were shared on Twitter between November 21, 2022 and January 10, 2023 and then calculated the engagement
30
they received on that platform.
For Facebook, we were then able to calculate likes and shares for those 44 links using CrowdTangle.
31
METHODOLOGY FOR CALCULATING MISINFORMATION AMPLIFICATION SCORE (MAF)
To measure the extent to which platforms amplify disinformation, we applied the methodology developed by the
Integrity Institute
to track the average Misinformation Amplification Factor (MAF) for social media platforms. The Integrity Institute also provided guidance and data processing support. The MAF is the ratio between how much engagement a disinformation post gets and what is the historical engagement of content from the creator.
The MAF for a specific piece of disinformation content is
-
Engagement on disinformation post / Average engagement on posts from creator prior to disinformation post
We compute the average engagement on all content from the creator who uploaded the disinformation content for the two weeks prior to them posting disinformation, or the 15 posts they made prior to the disinformation post when there is limited content in the two week window (in the rare case we canât get 15 pieces of content for the baseline, we remove that specific piece of misinformation content from consideration). This may include disinformation posts, if the creator is a serial poster of disinformation, which is one bias that can cause us to underestimate the MAF.
32
This gives us a MAF for every disinformation post that fact-checkers identify. To compute the MAF for each platform, we average the MAF for every post we collect on it. (Specifically, we average the logarithm of the MAF, since engagement on content generally follows a log-normal distribution.)
GENERAL NOTE ON METHODOLOGY
It is important to note that, while we collect data and compute numbers to the best of our ability, this analysis is not exhaustive as we looked only at a sample of fact-checked disinformation posts regarding the documentary
Died Suddenly.
Moreover, this research is made significantly more challenging because social media platforms do not provide investigators with access to the data needed to measure the total response rate, moderation speed, number of fact-checks and the amount of users who have seen or been targeted with disinformation.
We also recognise the hard work of employees at social media platforms across different sub-teams, who have done their best to push the companies to fix their disinformation problem. This report is not an indictment of their personal efforts, but rather highlights the need for much more proactive decisions and solutions implemented by the highest levels of executive power at each of the companies.
Cooperation across fields, sectors and disciplines is needed more than ever to fight disinformation. All social media platforms must become more transparent with their users and with researchers to ensure that the scale of this problem is measured effectively and to help public health officials respond in a more effectual and proportional manner to both the pandemic and the infodemic.
It is important to note that although fact-checks from reputable fact-checking organisations provide a reliable way to identify disinformation content, researchers and fact-checkers have a limited window into disinformation spreading in private groups, on private profiles and via messaging apps such as WhatsApp, Telegram or Facebook Messenger.
Similarly, engagement data for posts analysed in this study are only indicative of wider engagement with, and exposure to, disinformation. Consequently, the findings in this report are likely conservative estimates.
Tell Your Friends