Passive Threat Intelligence: Collecting Without Touching the Target

Before a threat intelligence team scans anything, it can learn a surprising amount about a domain from records other parties already keep. Certificate logs list its hostnames, registration data gives its dates and nameservers, and web archives show what it used to publish. For security teams, this passive stage is the cheapest and quietest part of the job, and it shows what any attacker can see too.

This guide covers the main passive sources, a small tested collector, and where “passive” stops being true. Use it on your own organisation or within an authorised scope.

What passive means

Passive collection means your requests go to a third party that already holds the data, and never to the target’s servers. MITRE ATT&CK describes the attacker’s version under Reconnaissance, including searching “freely available technical databases” (T1596), with entries for DNS and passive DNS, WHOIS, digital certificates, CDNs and scan databases.

MITRE also explains why defenders should care. It says this activity can take place outside the visibility of the target organisation, which makes it hard to detect, and that it cannot be easily mitigated with preventive controls. You cannot block someone reading a public log. What you can do is look first and reduce what the logs reveal.

The sources worth checking

Source	What it gives you	Its limits
Certificate Transparency logs	Hostnames on issued TLS certificates	Shows names, not whether they are live; wildcards hide subdomains
RDAP	Registrar, dates, nameservers	Personal data is usually redacted
Web archives	Old pages, URLs and hosts	Only what was captured, often years old
Passive DNS	Which IPs a name resolved to, and when	Mostly commercial, and coverage varies
Internet scan databases	Open ports and banners seen by someone else	Snapshots, and may be out of date

Certificate Transparency is the richest free source. RFC 9162 describes a protocol for “publicly logging the existence” of TLS certificates so that anyone can audit certificate authorities, and logs are append-only. That also means a certificate for staging.internal-tool.example.com stays visible, even if the host only lasted a week.

RDAP is the structured replacement for WHOIS. ICANN says that since 28 January 2025, generic top-level domain registries and registrars must provide RDAP and are no longer required to provide WHOIS, so new tooling should start there.

Our guide to what OSINT is and where its legal boundaries sit covers the wider discipline, including sources about people, which this guide leaves out.

A small passive collector

The script below asks three third-party services about a domain and never connects to the domain itself. It runs on Python 3.9 and later and needs only requests. It keeps going when a source is down, because free public services often are: while testing this, crt.sh and the Wayback Machine each returned errors and timeouts for several minutes at a time.

from __future__ import annotations

import sys
import time

import requests

TIMEOUT = (10, 60)
HEADERS = {"User-Agent": "passive-recon-demo/1.0"}


def get_json(url, params=None):
    """Every request goes to a third-party source, never to the target itself."""
    for attempt in range(3):
        try:
            r = requests.get(url, params=params, headers=HEADERS, timeout=TIMEOUT)
            r.raise_for_status()
            return r.json()
        except requests.RequestException:
            if attempt == 2:
                raise
            time.sleep(10 * (attempt + 1))  # free public services go down often


def in_scope(host, domain):
    """The domain or a real subdomain of it; CT names also hold labels and lookalikes."""
    return " " not in host and (host == domain or host.endswith("." + domain))


def certificate_names(domain):
    """Hostnames from Certificate Transparency logs, through crt.sh."""
    rows = get_json("https://crt.sh/", {"q": f"%.{domain}", "output": "json"})
    names = {n.strip().lower() for row in rows for n in row["name_value"].splitlines()}
    return sorted(n for n in names if "*" not in n and in_scope(n, domain))


def archived_hosts(domain, limit=500):
    """Hostnames the Wayback Machine has captured, without loading any live page."""
    rows = get_json("https://web.archive.org/cdx/search/cdx", {
        "url": domain, "matchType": "domain", "output": "json",
        "fl": "original", "limit": limit,
    })
    urls = [row[0] for row in rows[1:]]  # the first row is the header
    hosts = {u.split("//")[-1].split("/")[0].split(":")[0].lower() for u in urls}
    return sorted(h for h in hosts if in_scope(h, domain))


def registration(domain):
    """Registrar and key dates from RDAP, the structured successor to WHOIS."""
    data = get_json(f"https://rdap.org/domain/{domain}")
    dates = {e["eventAction"]: e["eventDate"][:10] for e in data.get("events", [])}
    return {"registered": dates.get("registration"), "expires": dates.get("expiration"),
            "nameservers": sorted(n["ldhName"].lower() for n in data.get("nameservers", []))}


def collect(domain):
    sources = {"certificates": certificate_names, "archive": archived_hosts,
               "rdap": registration}
    found, failed = {}, {}
    for name, fetch in sources.items():
        try:
            found[name] = fetch(domain)
        except (requests.RequestException, ValueError, KeyError) as exc:
            failed[name] = type(exc).__name__  # a source being down is a result too
    return found, failed


if __name__ == "__main__":
    domain = sys.argv[1] if len(sys.argv) > 1 else "example.com"
    found, failed = collect(domain)
    hosts = set(found.get("certificates", [])) | set(found.get("archive", []))
    only_ct = set(found.get("certificates", [])) - set(found.get("archive", []))
    only_archive = set(found.get("archive", [])) - set(found.get("certificates", []))
    print(f"{domain}: {len(hosts)} hostnames seen by third parties")
    print(f"  in certificates only: {sorted(only_ct)[:6]}")
    print(f"  in the archive only:  {sorted(only_archive)[:6]}")
    print(f"  registration: {found.get('rdap')}")
    print(f"  sources unavailable: {failed or 'none'}")

Output for example.com:

example.com: 7 hostnames seen by third parties
  in certificates only: ['dev.example.com', 'emudhra-lab.example.com', 'm.example.com', 'products.example.com', 'support.example.com']
  in the archive only:  []
  registration: {'registered': '1995-08-14', 'expires': '2027-08-13', 'nameservers': ['elliott.ns.cloudflare.com', 'hera.ns.cloudflare.com']}
  sources unavailable: none

Two details in that code came from testing. Certificate logs hold more than hostnames: an early version used a plain suffix check and let through a certificate label with spaces in it and a lookalike, m.testexample.com. That is why in_scope matches a whole subdomain. The other detail is that each source sees different things, so compare them rather than trusting one. A hostname only in old archives may be gone, and one only in recent certificates may be new.

Where passive stops being passive

The line is easier to cross than it looks.

• Opening a URL you found. Visiting a live hostname from your results sends the target a request from your IP.
• Archived pages with live resources. A saved page can still try to load images or scripts from the original site. Disable external loading, or read the raw response.
• DNS lookups. A query through a resolver can reach the target’s authoritative nameservers. Historical passive DNS data does not.
• Your queries are visible to the source. The service you ask learns what you are looking at. For sensitive investigations, consider what that reveals, and read its terms and rate limits.

Treat results as leads. Records go stale, certificates outlive hosts, and shared hosting makes IP history misleading.

What to do with the results

For defenders, the result is a list of what the outside world can already see. A sensible routine:

1. Compare against your asset inventory. A hostname you cannot match to an owner is the finding.
2. Check what your names give away. Names like vpn-old or jenkins-test in public logs help attackers. Consider wildcard certificates for internal naming where they are appropriate, while weighing their own risks.
3. Repeat it, and alert on what is new. Our guide to continuous external attack surface monitoring shows how to turn this into a diff that has owners.
4. Move to active checks deliberately. Verify your own assets directly. For anything outside your organisation, get authorisation first. Where threat research needs to see what a suspicious site shows real visitors in a given country, that is active work with its own rules, covered in our piece on residential proxies for threat intelligence.

The bottom line

Passive collection gives you a useful picture of a domain for almost no cost and without alerting it. Start with certificate logs, RDAP and archives, compare them, treat what you find as leads, and be honest about the moment your work stops being passive. Run it on yourself first, because anyone else can.

Sources and references

• MITRE ATT&CK, Reconnaissance (TA0043): attack.mitre.org
• MITRE ATT&CK, Search Open Technical Databases (T1596): attack.mitre.org
• RFC 9162, Certificate Transparency Version 2.0: rfc-editor.org
• Certificate Transparency, How CT works: certificate.transparency.dev
• RFC 9083, JSON Responses for RDAP: rfc-editor.org
• ICANN, Registration Data Access Protocol: icann.org
• Internet Archive, Wayback CDX Server API: github.com/internetarchive/wayback