Skip to Main content Skip to Navigation
Journal articles

Flagging incorrect nucleotide sequence reagents in biomedical papers: To what extent does the leading publication format impede automatic error detection?

Abstract : In an idealised vision of science the scientific literature is error-free. Errors reported during peer review are supposed to be corrected prior to publication, as further research establishes new knowledge based on the body of literature. It happens, however, that errors pass through peer review, and a minority of cases errata and retractions follow. Automated screening software can be applied to detect errors in manuscripts and publications. The contribution of this paper is twofold. First, we designed the erroneous reagent checking (ERC) benchmark to assess the accuracy of fact-checkers screening biomedical publications for dubious mentions of nucleotide sequence reagents. It comes with a test collection comprised of 1679 nucleotide sequence reagents that were curated by biomedical experts. Second, we benchmarked our own screening software called Seek&Blastn with three input formats to assess the extent of performance loss when operating on various publication formats. Our findings stress the superiority of markup formats (a 79% detection rate on XML and HTML) over the prominent PDF format (a 69% detection rate at most) regarding an error flagging task. This is the first published baseline on error detection involving reagents reported in biomedical scientific publications. The ERC benchmark is designed to facilitate the development and validation of software bricks to enhance the reliability of the peer review process.
Document type :
Journal articles
Complete list of metadatas

Cited literature [39 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02911605
Contributor : Guillaume Cabanac <>
Submitted on : Tuesday, August 4, 2020 - 10:06:33 AM
Last modification on : Tuesday, November 24, 2020 - 4:00:18 PM
Long-term archiving on: : Monday, November 30, 2020 - 2:00:35 PM

File

LabbeEtAl2020.pdf
Publisher files allowed on an open archive

Licence


Distributed under a Creative Commons Attribution - ShareAlike 4.0 International License

Identifiers

Citation

Cyril Labbé, Guillaume Cabanac, Rachael West, Thierry Gautier, Bertrand Favier, et al.. Flagging incorrect nucleotide sequence reagents in biomedical papers: To what extent does the leading publication format impede automatic error detection?. Scientometrics, Springer Verlag, 2020, 124 (2), pp.1139-1156. ⟨10.1007/s11192-020-03463-z⟩. ⟨hal-02911605⟩

Share

Metrics

Record views

73

Files downloads

60