True or false: studying work practices of professional fact-checkers

This research included Nasir Memon of NYU Tandon and the NYU Center for Cybersecurity, Nicholas Micallef of the NYU Abu Dhabi Center for Cyber Security, and researchers from Indiana University Bloomington and the University of Utah. 

Online misinformation is a critical societal threat . While fact-checking plays a role in combating the exponential rise of misinformation, little empirical research has been done on the work practices of professional fact-checkers and fact-checking organizations.

Existing research has covered fact-checking practitioner views, effectiveness of fact checking efforts, and professional and user practices for responding to political claims. While researchers are beginning to investigate challenges to fact-checking, such efforts typically focus on traditional media outlets rather than independent fact-checking organizations (e.g., Politifact). Similarly, such research has not yet investigated the entire misinformation landscape, including the dissemination of the outcomes of fact-checking work. 

To address these shortcomings, a team including Nasir Memon of NYU Tandon and Nicholas Micalleff of NYU Abu Dhabi interviewed 21 professional fact-checkers from 19 countries, covering topics drawn from previous research analyzing fact-checking from a journalistic perspective. The interviews focused on gathering information about the fact-checking profession, fact-checking processes and methods, the use of computation tools for fact-checking, and challenges and barriers to fact-checking. 

The study, "True or False: Studying the Work Practices of Professional Fact-Checkers," found that most of the fact-checkers felt they have a social responsibility of correcting harmful information to provide “a service to the public,” emphasizing that they want the outcome of their work to both educate and inform the public. Some fact-checkers mentioned that they hope to contribute to an information ecosystem providing a “balanced battlefield” for the discussion of an issue, particularly during elections.

The interviews also revealed that the fact-checking process involves first selecting a claim, contextualizing and analyzing it, consulting data and domain experts, writing up the results and deciding on a rating, and disseminating the report.

Fact-checkers encounter several challenges in achieving their desired impact because current fact-checking work practices are largely manual, ad-hoc, and limited in scale, scope, and reach. As a result, the rate at which misinformation can be fact-checked is much slower than the speed at which it is generated. The research points out the need for unified and collaborative computational tools that empower the human fact-checker in the loop by supporting the entire pipeline of fact-checking work practices from claim selection to outcome dissemination. Such tools could help narrow the gap between misinformation generation and fact-check dissemination by improving the effectiveness, efficiency, and scale of fact-checking work and dissemination of its outcomes. 

This research has been supported by New York University Abu Dhabi and Indiana University Bloomington.