Internet Archive & Wayback Machine Scraper - Snapshots & Search

List Wayback Machine snapshots of any URL or site with date, status and change filters and optional page text, or search archive.org item metadata. Official APIs, no key, $0.50 per 1,000 rows.

USD 0.50 per 1,000 results on Apify (pay per event)

Open in Apify StoreExample: Wayback Machine History of a Website, Yearly

Two tools in one actor, both on archive.org's official public APIs, with no API key, no login and no browser:

• Wayback Machine mode lists every archived snapshot of a page or a whole site: capture date, HTTP status, content type, content hash and a ready-to-open snapshot link. Filter by date, status and type, keep only content changes, and optionally download the archived page text.
• Search mode exports archive.org item metadata for books, audio, video, software and web collections, with optional file lists.

Price: $0.50 per 1,000 rows in either mode.

Common uses

• See how a competitor's pricing page, homepage or terms changed over time (collapse = content changes only, with page text).
• Recover the text of a page or site that is now offline.
• Check a domain's history before buying it: first and last capture, gaps, redirects and what it used to host.
• Find every archived URL on a site for SEO migrations, redirect maps or broken link recovery.
• Build reading lists or text corpora from public domain books, or export a collection's catalogue to a spreadsheet.

Quick start: Wayback Machine

{
  "mode": "wayback",
  "urls": ["example.com", "https://www.python.org/"],
  "collapse": "year",
  "from": "2015",
  "to": "2020",
  "statusFilter": "200",
  "maxSnapshotsPerUrl": 50,
  "fetchContent": true
}

Wayback inputs

Wayback output

{
  "type": "wayback_snapshot",
  "input_url": "https://www.python.org/",
  "original_url": "https://www.python.org/",
  "urlkey": "org,python)/",
  "timestamp": "20150101032028",
  "captured_at": "2015-01-01T03:20:28Z",
  "status_code": 200,
  "mimetype": "text/html",
  "digest": "CWG5GMVD...",
  "length": 6411,
  "snapshot_url": "https://web.archive.org/web/20150101032028/https://www.python.org/",
  "raw_snapshot_url": "https://web.archive.org/web/20150101032028id_/https://www.python.org/",
  "page_title": "Welcome to Python.org",
  "text": "Skip to content ...",
  "text_length": 6678,
  "content_error": null
}

page_title, text, text_length and content_error appear only when fetchContent is on. raw_snapshot_url opens the page exactly as archived, without the Wayback toolbar. length is the compressed size in the archive. Exact duplicate index entries are skipped and not charged. A per-target summary is saved to the run's OUTPUT record, and one failing URL does not stop the others.

Search mode

Search mode is the default, so existing inputs keep working unchanged.

{ "query": "mediatype:texts AND subject:python", "sort": "downloads desc", "maxItems": 100 }

Search inputs

Query syntax

Archive.org uses Lucene syntax. Some useful patterns:

subject:jazz AND mediatype:audio
creator:"Ernest Hemingway"
collection:gutenberg AND language:english
date:[2020-01-01 TO 2023-12-31]

Search output fields

Search sample output

Result from query mediatype:texts AND subject:python, sorted by downloads desc:

{"identifier":"python_ebooks_2020","title":"Python Ebooks","creator":["strugglez"],"description":"Python Ebooks","mediatype":"texts","subject":["python"],"date":null,"year":null,"language":["Latin"],"downloads":257128,"item_size":49856126105,"collection":["booksbylanguage_latin","booksbylanguage"],"publicdate":"2020-09-27T09:24:26Z","item_url":"https://archive.org/details/python_ebooks_2020","files":null}

Pricing

Pay per result: $0.50 per 1,000 rows, the same in both modes, plus Apify's tiny start fee. 100 snapshots cost $0.05. Downloading page text or file lists does not raise the per-row price, it only makes the run take longer.

Limitations

• Wayback mode lists what the Internet Archive has captured. Pages it never archived, and sites that asked to be excluded, will return no rows.
• collapse: digest compares each capture with the one before it in the index, so it removes repeats in a row, not every duplicate across years.
• Page text is plain text extracted from the archived HTML. It does not run JavaScript, so pages that load their content with scripts may come back mostly empty. Images, PDFs and other non-text files are listed but their content is not downloaded (content_error says not_text).
• With fetchContent on, each snapshot takes about 1.5 seconds, because the actor waits between requests to stay polite to archive.org. Use collapse or a small maxSnapshotsPerUrl for long histories.
• Archive.org caps search mode at 10,000 items per query. Split larger jobs by date or subject.
• Some search fields (date, year, language) are not consistently filled in by contributors, and collection includes personal favourites (prefixed fav-).

Rate limits

Requests run one at a time: 1 second between Wayback index pages, 1.5 seconds between page downloads, 500 ms between search pages and 600 ms between file list requests, with automatic backoff on HTTP 429 and 5xx responses.

Legal

Both modes use archive.org's public endpoints, which need no authentication. Only publicly archived data is returned. Respect the copyright of the archived pages you download.

Run it on Apify Apify's free plan includes monthly platform credit, so small runs cost little or nothing to try.