archivefetch

A fast, concurrent CLI tool to fetch archived URLs from the Wayback Machine CDX API.

Install

go install github.com/AliHzSec/archivefetch@latest

Usage

archivefetch -d example.com

Flags

Flag	Short	Default	Description
-domain	-d	—	Target domain (required)
-list	-l	—	File with list of domains (one per line)
-subs	-s	false	Include subdomains
-threads	-t	10	Concurrent fetch threads
-retry	-r	0	Max retries per request
-page-size	-ps	50	CDX page size (1–50)
-timeout	-T	60	HTTP timeout in seconds
-proxy	-p	—	HTTP/SOCKS5 proxy
-output	-o	stdout	Output file path
-silent	-S	false	Suppress logs, print URLs only

Examples

# Basic fetch
archivefetch -d example.com

# Include subdomains, save to file
archivefetch -d example.com -s -o example_urls.txt

# Multiple domains from file
archivefetch -l domains.txt -s -o results.txt

# With proxy and retries
archivefetch -d example.com -p "http://127.0.0.1:8080" -r 3

# Silent mode (pipe-friendly)
archivefetch -d example.com -S  |  grep  "\.php"

# Increase threads for faster fetching
archivefetch -d example.com -t 20 -o output.txt