Most competitor creative libraries die the same way. Collection works, assets accumulate, and after four months there are ninety thousand files that nobody can search, in which the same ad appears two hundred times because a tracking parameter changed, and which no strategist opens because finding anything takes longer than asking a colleague.
The engineering problem is not acquiring creatives. It is deduplication and metadata. Get those two right and a modest library is genuinely useful. Get them wrong and volume makes it worse, not better.
Collect from the permitted sources first
Before building anything, use the platform ad libraries. Meta, Google, TikTok and LinkedIn all publish searchable archives of active advertising, and they are the cheapest, most complete and most defensible source of competitor creative that exists.
They give you the asset, the advertiser, the run dates and often the surfaces, in a form that is intended to be queried. Start there, and build collection only for what they do not cover: display and native placements, retail media, affiliate content, email and newsletter placements, and landing-page assets.
That order matters commercially as well as legally. Engineering effort spent re-collecting what an archive already publishes is effort not spent on the parts nobody has.
Deduplication is the whole system
The same creative reaches you many times: re-served with different tracking parameters, resized for different placements, recompressed by a CDN, re-uploaded by the advertiser with no changes, or captured twice by your own collector.
Exact file hashing does not solve this, because a single byte of difference produces a different hash and CDNs produce those differences constantly. What works is a layered identity.
Perceptual hashing for images. A hash computed from image structure rather than bytes, so a resized or recompressed version of the same creative lands close to the original. Store the hash and group by distance threshold rather than equality.
Frame sampling for video. Perceptual hashes of frames at fixed intervals, plus duration. A video re-encoded at a different bitrate matches; a genuinely different cut does not.
Normalised copy text. The headline and body with case, punctuation and whitespace normalised. Copy is often the most stable identifier when the asset itself has been reworked.
A creative family, not a flat record. Group variants under a parent, and keep every variant linked to it. Variant count is a real signal, since a competitor producing forty variants of one concept is testing, and that is worth knowing.
Pick the distance threshold empirically by hand-reviewing a sample of matched and unmatched pairs, then record the resulting false-merge and false-split rates. Without those numbers nobody can size the error in a report built on the library.
Make it searchable, which means extracting text
An image nobody can search is a file. Most of the value in a creative library is in being able to ask which competitors are using a particular claim, and that requires the words.
Run OCR over image creatives to capture on-image copy, which is frequently where the actual offer lives. Transcribe video and audio. Then index the extracted text alongside the structured metadata. This one step is the difference between a library that gets used and an archive that gets forgotten.
Store the extracted text as its own field rather than overwriting anything, and keep the confidence score. OCR on stylised advertising copy is imperfect, and a low-confidence extraction should be visibly low-confidence rather than quietly wrong.
The metadata schema
The asset is the smaller half of a record.
| Field | Notes |
|---|---|
| Creative family ID | The dedupe parent, so variants group |
| Perceptual hash | Plus the algorithm and threshold used, so results stay reproducible |
| Advertiser as displayed | Which differs from the parent brand more often than expected |
| Surface and placement | Search, feed, display, native, retail media, email |
| Market | Country, and city where the placement is local |
| First and last seen | The run window as you observed it, not as claimed |
| Vantage point | Exit country and city the observation came from |
| Device profile | Desktop or mobile, since creative differs |
| Extracted text | OCR or transcript, with a confidence score |
| Landing URL and redirect chain | Where the offer actually lives |
| Source | Which ad library or which observed placement it came from |
First and last seen are what turn a library into a timeline. A creative that ran for four months is a proven performer; one that ran for six days was a failed test. That distinction is most of the strategic value and it costs one field.
The collection layer
Two properties matter for the parts the ad libraries do not cover.
Display and native creative is targeted, so the vantage point determines what you are served at all. Residential proxies with country targeting let each market’s collection come from that market. With the Shifter gateway, targeting and session go in the credentials against p.shifter.io:443:
customer-USERNAME-country-de-sid-creative-de-07-ttl-600:PASSWORDcountry-de sets the market and sid-creative-de-07 holds one exit across a collection pass, so the creatives in a snapshot belong to one coherent session rather than several vantage points mixed together.
The second property is bandwidth, and creative collection is unusually heavy because the payload is the point. Images and video are the asset, so the usual advice to block them does not apply. Instead, control volume: deduplicate before downloading wherever the source exposes an asset URL you can hash or compare against what you already hold, skip re-fetching a creative family you already have variants of, and cap resolution where a thumbnail answers the question. Sizing the pipeline is covered in forecasting residential proxy bandwidth, and the cost levers in cutting proxy bandwidth costs.
Keep request rates ordinary with real backoff, as in rate limiting and request throttling.
Copyright, plainly
A competitor’s creative is their copyrighted work. Collecting it for internal competitive analysis is ordinary business research. Republishing it is not.
The practical rules that keep a library defensible: keep it internal, behind access control, and do not surface it in anything customer-facing or public. Do not use a competitor’s asset in your own advertising, on your website, or in published content. Attribute in internal decks and keep the source and date on every record. Set a retention period rather than keeping everything forever. And take advice before any external use, including in a sales deck that leaves the building.
Do not click competitor ads to reach their creative or landing page. A click costs them money, which turns observation into interference; resolve the destination from the markup and redirect chain instead. The wider framing is in residential proxies for competitor ad intelligence.
What a working library is used for
Four questions justify the build, and they are all queries over metadata rather than browsing over assets.
Which claims are competitors making, and where. A text search across extracted copy, sliced by market.
What survives. Creatives ranked by observed run length, which is the closest public proxy for what performs.
What is being tested. Variant counts per creative family, and how quickly new families appear.
When positioning moved. A timeline of first-seen dates for families carrying a new claim, which frequently shows a market-by-market rollout.
FAQ
How far back should a library go?
Enough to establish what persists, which usually means a year. Retention beyond that rarely changes a decision and increases the copyright surface you are holding.
Should I store the assets or just the URLs?
Store the assets. Creative URLs expire, and a library of dead links is not a library. Store the URL as well, for provenance.
What is the most common reason these projects fail?
No deduplication, followed by no text extraction. The first makes the library unusable by volume, the second makes it unsearchable.
Can I show competitor creatives in a public blog post or ad?
Not without advice. Internal analysis is the safe use. Publication is someone else’s copyright decision, not yours.
The bottom line
A competitor creative library is a metadata product with files attached. Query the official ad libraries first, collect only what they do not cover, give every asset a perceptual identity so variants group instead of piling up, extract the text so the thing is searchable, and record first and last seen so you can tell a proven creative from a failed test.
Keep it internal, keep the provenance, and treat someone else’s creative as someone else’s property. The product view is on the marketing and adtech page, with rates on the pricing page.