Mozilla commissions CheckFirst to conduct a stress test of Ad Repositories

This report examines the ad transparency tools of 11 of the world’s largest tech companies, looking particularly at their usability for researchers. None of the scrutinised platforms completely achieved our baseline criteria which we developed for this “stress test”. 

The need for transparency in online advertising has never been more critical as nefarious use of ads is on the rise (see our Facebook Hustles investigation, AI Forensic paper, EU DisinfoLab, Reset, Stiftung Neue Verantwortung, and Qurium reports) and they can be used for destabilisation attempts. Ad libraries, which are now mandated by the DSA for designated services, should enable researchers and users to understand and analyse who is behind an ad, how and who it targets. Not only did we find that no platform is currently delivering fully adequate tools to achieve these goals, but each and every company presents data in a different format, complicating comparisons across datasets. This is why Check First proposes the adoption of the Standardised Digital Advertising Format (SDAF), a common format for ad data.

What we found in our research: tech giants’ ad transparency tools fail to meet our expectations

The research we conducted revealed a stark reality: the ad transparency tools developed by the major platforms are failing to meet their intended purpose. Let’s face it, our study, Full Disclosure: Stress-testing tech platforms’ ad repositories, paints a bleak picture. A striking example? Only 5 platforms out of the 14 analysed allow users to find their way to the ad library from their main interface. These tools are supposedly designed to help watchdogs monitor ad content across platforms and to enable researchers to monitor a wide range of issues such as election disinformation. These are critical tools for  the research community to raise awareness about scams, misleading ads, and ads that may violate election rules or terms and conditions of the platforms like political ads on TikTok. Given the audiences of these major platforms, you can only imagine the damage this can do, especially in this election year when almost half of the world’s population will vote.

Who did we investigate?

We examined AliExpress, Apple App Store, Bing, Booking.com, Alphabet’s Google Search & YouTube, LinkedIn, Meta, Pinterest, Snapchat, TikTok, X, and Zalando. CheckFirst researchers evaluated these ad libraries based on over 20 parameters, including functionality, data accessibility and accuracy, inspired by both the DSA rules and Ad API guidelines created by Mozilla and other researchers in 2019.

None of the advertising transparency tools studied were found to be sufficiently effective, leaving voters around the world vulnerable to manipulation and misinformation. The implications extend beyond the borders of the European Union, potentially affecting countries such as the United States, Mexico and India, all of which have major elections in the near future. For example, X, formerly known as Twitter, is particularly disappointing. X’s transparency tool offers no filtering and sorting capabilities, ads can only be accessed through a cumbersome CSV export file; the content of ads is not disclosed (only a URL to the ads), and there are gaps in targeting parameters and recipient data:

“In order to search for a specific ad, the user needs to know who placed the ad, in which country, and when. All three pieces of information have to be known, which makes natural search impossible”  

Amaury L., Chief Technology Officer and co-founder of CheckFirst

While some platforms such as Apple, LinkedIn and TikTok fared slightly better, they too had significant gaps in data and functionality. Issues such as search rate limitations, poor sorting and filtering capabilities, and limited accessibility were common across most platforms.

Amaury L. emphasised the importance of transparency in advertising on big platforms, especially in areas that affect the public interest, such as fair elections and public health. 

“Who’s paying for ads and how they’re targeted is critical to helping watchdogs look out for the public interest – whether it’s fair elections, public health or social justice. In short, if you see an ad telling you that climate change is a hoax, you might be interested to know if that ad is paid for by the fossil fuel industry.”

Additional findings from the study underscored missing ads, inconsistent public accessibility across platforms, and a lack of adequate search features. Furthermore, paid influencer content or ‘branded content’ raised concerns about underreporting and transparency in this domain.

Our key recommendation: a common and standardised tool for ad libraries

Based on these worrying findings, CheckFirst put forward the Standardised Digital Advertising Format (SDAF). The goal of this data framework is to enable comparative research across ad data across platforms.

Our goal is to help the stakeholders  to create accessible, logical, interoperable ad repositories. We firmly believe that our open source standard can contribute to such an endeavour, within the strong framework of the DSA. We are very proud that this tool is supported by organisations like Reset, AI Forensics, Who Targets Me, Check My Ads, SNV.

Only through collaborative efforts can we mitigate the harm made by the lack of transparency. This is precisely what this study is about: the urgent need for improved accountability and oversight mechanisms in online advertising by the largest online platforms.