Subdomains Enumeration Cheat Sheet

Hi, this is a cheat sheet for subdomains enumeration.

I will update it every time I find a new interesting tool or technique. So keep an eye on this page!

subdomains-enumeration-cheatsheet.png

Table of contents

Why so many tools & techniques?

  • The more techniques used, the more chances to find interesting subdomains that others might have missed.
  • Some bug hunters recommend using only a handful of tools (like Amass, Massdns, Subfinder & Gobuster). But people who have a bad Internet connection & no VPS won’t be able to use these highly effective & fast tools. So choose whatever works for you!

Methods

  • Scraping
  • Brute-force
  • Alterations & permutations of already known subdomains
  • Online DNS tools
  • SSL certificates
  • Certificate Transparency
  • Search engines
  • Public datasets
  • DNS aggregators
  • Git repositories
  • Text parsing (HTML, JavaScript, documents…)
  • VHost discovery
  • ASN discovery
  • Reverse DNS
  • Zone transfer (AXFR)
  • DNSSEC zone walking
  • DNS cache snooping
  • Content-Security-Policy HTTP headers
  • Sender Policy Framework (SPF) records
  • Subject Alternate Name (SAN)

Linux tools

AltDNS

  • Description
  • Installation
    git clone https://github.com/infosec-au/altdns.git
    cd altdns
    pip install -r requirements.txt
    
  • Usage:
    • Generate a list of altered subdomains: ./altdns.py -i known-subdomains.txt -o new_subdomains.txt
    • Generate a list of altered subdomains & resolve them: ./altdns.py -i known-subdomains.txt -o new_subdomains.txt -r -s resolved_subdomains.txt

    • Other options
      • -w wordlist.txt: Use custom wordlist (default altdns/words.txt)
      • -t 10 Number of threads
      • -d $IP: Use custom resolver

Amass

  • Description
  • Installation
    • go get -u github.com/OWASP/Amass/...
  • Usage
    • Get target’s ASN from http://bgp.he.net/
    • amass -d target.com -o $outfile
    • Get subdomains from ASN: amass.netnames -asn $asn

Assets-from-spf

  • Description
  • Installation
    git clone https://github.com/yamakira/assets-from-spf.git
    pip install click ipwhois
    
  • Usage
    • cd the-art-of-subdomain-enumeration; python assets_from_spf.py target.com
    • Options
      • --asn: Enable ASN enumeration

BiLE-suite

  • Description
  • Installation
    aptitude install httrack
    git clone https://github.com/sensepost/BiLE-suite.git
    
  • Usage
    • List links related to a site: cd BiLE-suite; perl BiLE.pl target.com target
    • Extract subdomains from the results of BiLe.pl: ` cat target.mine grep -v “Link from” cut -d’:’ -f2 grep target.com sort uniq`

Bing

  • Search engine
  • Usage
    • Find subsomains: site:target.com
    • Find subdomains & exclude specific ones: site:target.com -site:www.target.com

Censys_subdomain_enum.py

Cloudflare_enum.py

Crt_enum_psql.py

Crt_enum_web.py

CTFR

  • Description
  • Installation
    git clone https://github.com/UnaPibaGeek/ctfr.git
    cd ctfr
    pip3 install -r requirements.txt
    
  • Usage
    • cd ctfr; python3 ctfr.py -d target.com -o $outfile

Dig

  • Description
    • Zone transfer, DNS lookups & reverse lookups
  • Installation
    • Installed by default in Kali, otherwise:
    • aptitude instal dnsutils
  • Usage dig +multi AXFR target.com dig +multi AXFR $ns_server target.com

Domains-from-csp

  • Description
  • Installation
    git clone https://github.com/yamakira/domains-from-csp.git
    pip install click
    
  • Usage
    • Parse CSP header for domains: cd domains-from-csp; python csp_parser.py $URL
    • Parse CSP header & resolve the domains: cd domains-from-csp; python csp_parser.py $URL -r

Dnscan

  • Description
  • Install
    git clone https://github.com/rbsec/dnscan.git
    cd dnscan
    pip install -r requirements.txt
    
  • Usage
    • Subdomain brute-force of a domain: dnscan.py -d target.com -o outfile -w $wordlist
    • Subdomain brute-force of domains listed in a file (one by line): dnscan.py -l $domains_file -o outfile -w $wordlist
    • Other options:
      • -i $file: Output discovered IP addresses to a text file
      • -r: Recursively scan subdomains
      • -T: TLD expansion

Dnsrecon

  • Description
    • DNS zone transfer, DNS cache snooping, TLD expansion, SRV enumeration, DNS records enumeration, brute-force, check for Wildcard resolution, subdomain scraping, PTR record lookup, check DNS server cached records, mDNS records enumeration…
    • https://github.com/darkoperator/dnsrecon
  • Installation
    • aptitude install dnsrecon on Kali, or:
      git clone https://github.com/darkoperator/dnsrecon.git
      cd dnsrecon
      pip install -r requirements.txt
      
  • Usage
    • Brute-force: dnsrecon -d target.com -D wordlist.txt -t brt
    • DNS cache snooping: dnsrecon -t snoop -D wordlist.txt -n 2.2.2.2 where 2.2.2.2 is the IP of the target’s NS server
    • Options
      • --threads 8: Number of threads
      • -n nsserver.com: Use a custom name server
      • Output options
        • --db: SQLite 3 file
        • --xml: XML file
        • --json: JSON file
        • --csv: CSV file

Dnssearch

  • Description
  • Installation
    go get github.com/evilsocket/dnssearch
    
    • Add ~/go/bin/ to PATH by adding this line to ~/.profile: export PATH=$PATH:/home/mima/go/bin/
  • Usage
    • dnssearch -domain target.com -wordlist $wordlist
    • Other options
      • -a bool: Lookup A records (default true)
      • -txt bool: Lookup TXT records (default false)
      • -cname bool: Show CNAME records (default false)
      • -consumers 10: Number of threads (default 8)

Domained

  • Description
  • Installation
    git clone https://github.com/cakinney/domained.git
    cd domained
    pip install -r ./ext/requirements.txt
    python domained.py --install
    
  • Usage
    • Run Sublist3r (+subbrute), enumall, Knock, Amass & SubFinder: python domained.py -d target.com
    • Run only Amass & Subfinder: python domained.py -d target.com --quick
    • Brute-force with massdns & subbrute with Seclist wordlist, plus Sublist3r, Amass, enumall & SubFinder: python domained.py -d target.com --b
    • Bruteforce with Jason Haddix’s All.txt wordlist, plus Sublist3r, Amass, enumall & SubFinder: python domained.py -d target.com -b --bruteall
    • Other options
      • --notify: Send Pushover or Gmail notifications
      • --noeyewitness: No Eyewitness
      • --fresh: Delete old data from output folder

Fierce

Gobuster

  • Description
  • Installation
    git clone https://github.com/OJ/gobuster.git
    cd gobuster/
    go get && go build
    go install
    
  • Usage
    • gobuster -m dns -u target.com -w $wordlist
    • Other options:
      • -i: Show IP addresses
      • -t 50: Number of threads (default 10)

Google

  • Search engine
  • Usage
    • Find subsomains: site:*.target.com
    • Find subdomains & exclude specific ones: site:*.target.com -site:www.target.com -site:help.target.com

Knock

  • Description
  • Install
    apt-get install python-dnspython
    git clone https://github.com/guelfoweb/knock.git
    cd knock
    nano knockpy/config.json # <- set your virustotal API_KEY
    python setup.py install
    
  • Usage
    • Use default wordlist: knockpy target.com
    • Use custom wordlist: knockpy target.com -w $wordlist
    • Resolve domain name & get response headers: knockpy -r target.com or knockpy -r $ip
    • Save scan output in CSV: knockpy -c target.com
    • Export full report in JSON: knockpy -j target.com

Ldns-walk

  • Description
    • DNSSEC zone walking
  • Installation
    • aptitude install ldnsutils
  • Usage
    • Detect if DNSSEC NSEC or NSEC3 is used:
      • ldns-walk target.com
      • ldns-walk @nsserver.com target.com
    • If DNSSEC NSEC is enabled, you’ll get all the domains
    • If DNSSEC NSEC3 is enabled, use Nsec3walker

Massdns

  • Description
  • Installation
    git clone https://github.com/blechschmidt/massdns.git
    cd massdns/
    make
    
  • Usage
    • Resolve domains: cd massdns; ./bin/massdns -r lists/resolvers.txt -t AAAA -w results.txt domains.txt -o S -w output.txt
    • Subdomain brute-force: ./scripts/subbrute.py wordlist.txt target.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w output.txt
    • Get subdomains with CT logs parser & resolve them with Massdns: ./scripts/ct.py target.com | ./bin/massdns -r lists/resolvers.txt -t A -o S -w output.txt
    • Other options:
      • -s 5000: Number of concurrent lookups (default 10000)
      • -t A (default), -t AAAA, -t PTR…: Type of DNS records to retrieve
      • Output options
        • -o S -w output.txt: Save output as simple text
        • -o F: Save output as full text
        • -o J: Save output as ndjson

Nsec3walker

  • Description
  • Installation
    wget https://dnscurve.org/nsec3walker-20101223.tar.gz
    tar -xzf nsec3walker-20101223.tar.gz
    cd nsec3walker-20101223
    make
    
  • Usage
    ./collect target.com > target.com.collect
    ./unhash  target.com.collect > target.com.unhash
    cat target.com.unhash | grep "target" | wc -l
    cat target.com.unhash | grep "target" | awk '{print $2;}'
    

Rapid7 Forward DNS dataset (Project Sonar)

  • Description
  • Installation
    • aptitude install jq pigz
  • Usage
    wget https://scans.io/data/rapid7/sonar.fdns_v2/20170417-fdns.json.gz
    cat 20170417-fdns.json.gz | pigz -dc | grep ".target.org" | jq`
    

San_subdomain_enum.py

Second Order

  • Description
  • Installation
    • go get github.com/mhmdiaa/second-order
  • Usage
    • Create a new copy of the default config.json file: cp ~/go/src/github.com/mhmdiaa/second-order/config.json ~/go/src/github.com/mhmdiaa/second-order/config-subs-enum.json
    • And edit ` ~/go/src/github.com/mhmdiaa/second-order/config-subs-enum.json to replace “LogCrawledURLs”: false with “LogCrawledURLs”: true`
    • second-order -base https://target.com -config config.json -output target.com
    • Look for new subdomains in the resulting folder (./target.com)

Subbrute

  • Description
  • Installation
    aptitude install python-dnspython
    git clone https://github.com/TheRook/subbrute.git
    
  • Usage
    • Test a single domain: ./subbrute.py target.com
    • Test multiple domains: ./subbrute.py target1.com target2.com
    • Test a list of domains: ./subbrute.py -t domains.txt
    • Enumerate subdomains, then their own subdomains:
      ./subbrute.py target.com > target.out
      ./subbrute.py -t target.out
      
    • Other options
      • -s wordlist.txt: Use a custom subdomains wordlist
      • -p: Print data from DNS records
      • -o outfile.txt: Save output in Greppable format
      • -j JSON: Save output to JSON file
      • -c 10: Number of threads (default 8)
      • -r resolvers.txt: Use a custom list of DNS resolvers

Subfinder

  • Description
  • Installation:
    • go get github.com/subfinder/subfinder
    • Configure API keys: ./subfinder --set-config VirustotalAPIKey=0x41414141
  • Usage
    • Scraping: ./subfinder -d target.com -o $outfile
    • Scraping & brute-force: subfinder -b -d target.com -w $wordlist -o $outfile
    • Brute-force only: ./subfinder --no-passive -d target.com -b -w $wordlist -o $outfie
    • Other options:
      • -t 100: Number of threads (default 10)
      • -r 8.8.8.8,1.1.1.1 or -rL resolvers.txt: Use custom resolvers
      • -nW: Exclude wildcard subdomains
      • -recursive: Use recursion
      • -o $outfile -oJ: JSON output

Sublist3r

  • Description
  • Installation
    git clone https://github.com/aboul3la/Sublist3r.git
    cd Sublist3r
    pip install -r requirements.txt
    
  • Usage
    • Scraping: ./sublist3r.py -d target.com -o $outfile
    • Bruteforce: ./sublist3r.py -b -d target.com -o $outfile
    • Other options:
      • -p 80,443: Show only subdomains which have open ports 80 and 443

Theharvester

  • Description
    • Tool for gathering subdomain names, e-mail addresses, virtual hosts, open ports/ banners, and employee names from different public sources
    • Scraping, Brute-force, Reverse DNS, TLD expansion
    • Scraping sources: Threatcrowd, Crtsh, Google, googleCSE, google-profiles, Bing, Bingapi, Dogpile, PGP, LinkedIn, vhost, Twitter, GooglePlus, Yahoo, Baidu, Shodan, Hunter
    • https://github.com/laramies/theHarvester
  • Installation
    • aptitude install theharvester
  • Usage
    • Scraping: theharvester -d target.com -b all
    • Other options:
      • -h output.html: Save output to HTML file
      • -f output.html: Save output to HTML & XML files
      • -t: Also do TLD expansion discovery
      • -c: Also do subdomain bruteforce
      • -n: Also do a DNS reverse query on all ranges discovered

vhost-brute

  • Description
  • Installation
    aptitude install php-curl
    git clone https://github.com/gwen001/vhost-brute.git
    
  • Usage
    • php vhost-brute.php --ip=$ip --domain=target.com --wordlist=$outfile
    • Other options:
      • --threads=5: Maximum threads (default 1)
      • --port: Set port
      • --ssl: Force SSL

Virtual-host-discovery

  • Description
  • Installation
    • git clone https://github.com/jobertabma/virtual-host-discovery.git
  • Usage
    • cd virtual-host-discover; ruby scan.rb --ip=1.1.1.1 --host=target.com --output output.txt
    • Other options
      • --ssl=on: Enable SSL
      • --port 8080: Use a custom port
      • --wordlist wordlist.txt: Use a custom wordlist

Virustotal_subdomain_enum.py

Online tools

Search engines

Specialized search engines

Certificate transparency

Public datasets

Online DNS tools & DNS aggregators

Git repositories

Wordlists

Resources


Let me know if you have any comments, requests, questions… Feedback is always welcome.

See you next time!


Comments