Check First helped to design and follow the guidelines of the EFCSN for OSINT operations. Using the words of the authors of the guidelines: “In recent years, online investigations using openly available data has flourished. Inspired by the work of Bellingcat and other global leaders, OSINT has raised in many online conversations. Sometimes, it has also been used as a buzzword covering all type of online investigation. With an OSINT community being built, and the lack of common standards, multiple OSINT practitioners felt it was time to put together conceptions, methodologies, and good practices of their OSINT work. For practical reasons, they decided to focus their work on public-facing outcomes of OSINT.”
The process starts once an investigation paper draft is ready. Evaluation begins with a double assessment by two distinct internal auditors from the Check First staff. They will go through a checklist of 18 items (see here) and annotate it. If all items are checked, the assessment score will be 100% and the score will decrease as less items are checked.
Four item categories are examined for the assessment of any of our OSINT operations:
Once this internal review step is done, the draft is sent to two different contracted external auditors for a first assessment. These auditors are chosen based on their qualification on the topic of the investigation. A contract is signed each time in between Check First and each external expert, including an independence and impartiality clause. This clause guarantees that the experts will assess the paper regardless of any possible compensation from Check First for the reviewers’ time.
After a first external review, Check First may modify the draft to take into account remarks from the external auditors. If this is the case, a second and final assessment is performed by the external auditor who raised issues, or by both if both raised issues.
Once all reviews are done and the paper is ready to be published, the final assessment score of the paper is calculated from the average of all reviews. This score is intended to communicate to the reader how well our standards were observed. For any OSINT investigation to be published, this score must be above 80%.
The readers can also get information about the process and its auditors as it is required that the qualities of internal and external reviewers are included in the published paper, as well as the final assessment score and a link to the assessment review template document. Reviewers sign the exported pdf. The final document includes a version control table showing version numbers increased upon each modification and an annotation of the changes made.
Our hope is to communicate to the best of our ability the trustworthiness of our work and to prevent partiality or bias.
We empower the fight against disinformation. Check First is a software and methodologies company founded by a developer and two journalists. We focus on creating the best tools and practices for the peculiar job of countering and monitoring fakes and influence campaigns, and gain time in the process by fostering collaboration. Our best asset is the complementarity of our founding team, making us approach problems in a comprehensive and innovative way.Our story