Conference notes: Esoteric subdomain enumeration techniques (LevelUp 2017)

Hi, these are the notes I took while watching the “Esoteric subdomain enumeration techniques” talk given by Bharath Kumar on LevelUp 2017.

Links

• Video: https://youtu.be/e_Gq99CKAys
• Slides: https://speakerdeck.com/yamakira/esoteric-sub-domain-enumeration-techniques
• Github repository: https://github.com/appsecco/bugcrowd-levelup-subdomain-enumeration
• Cheatsheet: https://github.com/appsecco/bugcrowd-levelup-subdomain-enumeration/blob/master/cheatsheet.pdf

Common subdomain enumeration techniques:

• Google dorking
• Using specialized search engines like VirusTotal
• Dictionary based enumeration
• Subdomain bruteforce
• ASN discovery

Esoteric subdomain enumeration techniques

• Certificate transparency (CT)
• DNSSEC zone walking
• DNS zone transfer
• Passive recon using public datasets

Certificate transparency

• Official list of log files: https://www.certificate-transparency.org/known-logs
• CT logs search engines, useful for searching SSL/TLS certificates issued for a domain:
  • https://crt.sh/
  • https://censys.io/
  • https://google.com/transparencyreport/https/ct/
• Tools
  • subdomain_enum_crtsh.py
    • https://github.com/appsecco/bugcrowd-levelup-subdomain-enumeration/blob/master/subdomain_enum_crtsh.py
    • Uses crt.sh to search through CT logs for a given domain and extracts subdomains and emails
    • python subdomain_enum_crtsh.py --domains <target_domain>
  • subdomain_enum_censys.py
    • https://github.com/appsecco/bugcrowd-levelup-subdomain-enumeration/blob/master/subdomain_enum_censys.py
    • Does the same thing but uses Censys.io
    • python subdomain_enum_censys.py <target_domain>
    • python subdomain_enum_censys.py ietf.org | grep ".ietf." (to eliminate alias subdomains like cloudflare)

DNSSEC

Zone walking NSEC

• Zone walking using NSEC records is similar to DNS zone transfer
• It extracts all subdomains of a given domain from its NSEC records
• Tools:
  • ldns-walk
    • aptitude install ldnsutils (to install ldns-walk)
    • ldns-walk <target_domain>
    • ldns-walk @ns-server <target_domain>
  • dig
    • dig +short NSEC <target_domain> | awk '{print $1;}'

Zone walking NSEC3

• Zone walking using NSEC3 records
• Difference between NSEC & NSEC3:
  • NSEC3 records provide a signed gap of hashes of domain names
  • NSEC records provide a signed gap of domain names
• Steps for NSEC3 zone walking:
  • Collect NSEC3 hashes of a domain
  • Crack the hashes offline
• Tools:
  • nsec3walker
    • Installation:

        wget https://dnscurve.org/nsec3walker-20101223.tar.gz
        tar -xzf nsec3walker-20101223.tar.gz
        cd nsec3walker-20101223
        make
      

    • Usage:

        ./collect icann.org > icann.org.collect					# To collect hashes
        ./unhash < icann.org.collect > icann.org.unhash				# To crack hashes
        cat icann.org.unhash | grep "icann" | awk '{print $2;}' | sed 's/\.$//'	# To extract subdomains found
        cat icann.org.unhash | grep "icann" | wc -l				# To get the number of subdomains found
      

  • nsec3map
  • Hashcat or JohnTheRipper (to crack the hashes)

DNS zone transfer

• Tool:
  • dig
    • dig AXFR @ns1.iitk.ac.in. iitk.ac.in
• Common in internal networks
• Sometimes mitigations like IP-base filtering are used to restrict access to DNS zone transfer based on IPs. But this can be bypassed: On internal pentests, pretend to be the secondary nameserver by spoofing its IP address, initiate a zone transfer & sniff the zone data

Passive reconnaissance

Using public datasets

• Scans.io & Project Sonar by Rapid7 gather Internet wide scan data including port scans & DNS records
• There are a lot of data sets (listed on https://scans.io & https://opendata.rapid7.com
• E.g. Rapid7 Forward DNS
  • wget https://opendata.rapid7.com/sonar.fdns_v2/2018-03-31-1522483201-fdns_any.json.gz
• Extract subdomains of a given domain from a gz file:
  • zcat <dataset_name> | jq -r 'if (.name | test("\\.example\\.com$")) then .name else empty end'

Using Cloudflare

• This technique only works if the domain you’re targeting is not using Cloudflare
• Manual process
  • Log into Cloudflare https://www.cloudflare.com/a/login
  • “Add site” to your account
  • Provide the target domain as a site you want to add & click “Next” twice
  • Wait for Cloudflare to dig through DNS data & display the subdomains
• Tool to automate it:
  • cloudflare_enum.py
    • https://github.com/appsecco/bugcrowd-levelup-subdomain-enumeration/blob/master/cloudflare_enum.py
    • python cloudflare_enum.py <cloudflare_email> <target_domain>
    • cat output.csv | cut -d "," -f1 | uniq

See you next time!