Conference notes: Modern Pentest Tricks For Faster, Wider, Greater Engagements (Area 41 & HITB 2018)

Hi, these are the notes I took while watching the “Modern Pentest Tricks For Faster, Wider, Greater Engagements” talk given by Thomas Debize on both Area 41 & HITB 2018 conferences.

Links #

• Area 41 2018 talk
• HITB 2018 talk
• Slides (HITB 2018)
• Thomas Debize’s Github repository

What changed and why you need to adapt? #

In 2018 it is easily possible to: #

• Scan the entire IPv4 space in few minutes/hours/days
  • With distributed (vulnerable) computing: Census 2012
  • With asynchronous programming: ZMap, Masscan, Unicornscan
  • With third-party platforms doing it for you, sometimes for free: Shodan, ZoomEye (the chinese Shodan, less restriction free, good data quality), Scans.io, Censys.io
• Query all OSINT information you want
  • Offline, by building your own platform: Modern Internet Scale Reconnaissance
  • Online, by querying services like: Recon-ng, DomainTools (not cheap but good data quality), Pastebin, Certificate Transparency…
• Pwn large Windows corporate infrastructures
  • Starting with reconnaissance
    • of assets: PowerView
    • of admins: BloodHound
  • Then through exploitation: CrackMapExec, Responder
  • And just after, post-exploitation: Mimikatz, Invoke-Mimikatz (stealthy & partial version of Mimikatz), Empire
  • Or…just automate everything in one tool: Deathstar

Why do you need to adapt your techniques? #

Because:

• More & more security tools are writing more & more good quality & reliable tools
• You will be asked to cover wider scopes faster
• It has already changed

Current pentest assessment require pentesters to SCALE!

Modern tricks for modern pentesters #

Data analysis & processing: CSV #

• Pentest involves a lot of iterative work: scan & exploit targets, harvest data & use it on new & old targets
• So to scale, you need a good data format to quickly process new data on all pentest phases (recon, exploitation, post-exploitation & reporting)
• Use CSV for inputs & outputs. It is the best format for data analysis & processing
• It’s a simple human-readable format but there is no standard for CSV
• CSV just means separate some stuff with another stuff (a delimiter like a comma)
• Best practices:
  • Encoding: Use utf-8
    • Python 2 “csv” module doesn’t support utf-8, use unicodecsv instead
  • Quoting & escaping: choose to have all fields quoted to prevent CSV injection (Example)

Pentest/infosec tools offering CSV output #

• nmaptocsv
• wfuzz
• testssl.sh
• BloodHound
• Ophcrack
• Recon-ng
• Nikto
• Nessus

Tools to handle CSV #

• Microsoft Excel
  • Use the “Text to Columns” function to convert text to Excel columns & choose your delimiter, then use the “Filter” function
  • Con: Max number of lines is 1 million (~10 to 30 MB file, easily attainable, commonly encountered issue)
• csvkit
  • Free, open source, one of the best tools for CSV
  • Command-line suite
  • csvkit tools in a nutshell:

• Dataiku Data Science Studio (DSS)
  • Free & Enterprise editions, not Open Source
  • The free edition is enough
  • Allows to perform the same kind of processing than Excel but without size limitation
  • Intuitive, user-friendly & efficient
    • Only 4 hours on a 4 cores + 16 GB RAM machine to join the “hash” column a 30 GB uncompressed DB dump with a 4 GB “hash : cleartext” file
  • Official tutorials

Parallel execution: GNU Parallel #

• Pentest involves a lot of parallel work:
  • Extracting the results of a tool output on multiple targets
  • Launching the same bruteforce on multiple targets
• Parallel execution is crucial to scale on wide scopes
• GNU Parallel is a Perl script to parallelize any command and maximize I/O & CPU usage
• Useful options:

| — | — | | --progress | a percentage of done/to be done | | --bar | a nice progress bar | | --joblog | a log of executed tasks, allows resuming | | -- resume | resume to your current execution status | | --sshlogin | distribute the tasks on remote computers through SSH |

Example 1: Directory bruteforce #

• Run wfuzz on target_list.txt

$ parallel -a target_list.txt --joblog joblog --progress --bar 
	"wfuzz
		-f 'results/result_wfuzz_{= s/[:\/]/_/g =}.json',json
		--filter 'c<403'
		-R 3 -Z -c
		-z file, '/usr/share/dirbuster/wordlists/directory-list-lowercase-2.3-small.txt
		{1}/FUZZ"

- `{= s/[:\/]/_/g}` is a Perl/sed expression to remove bad chars in filename
- Format of *target_list.txt*:

http://foo
https://foobar
https://bar:443

• Then filter results by 200 HTTP response code & get a screenshot of all responding URLs with webscreenshot

$ ls ./results/result_wfuzz_*.json | parallel
"cat {} | jq '.[] | if (.code == 200) then .url else empty end' | sed 's/"//g' >> list_to_webscreenshot.txt"

Example 2: DNS enumeration #

• Reference: http://mubix.github.io/blog/2014/01/29/hostname-bruteforcing-on-the-cheap/
• Run dig on subdomains.txt

$ parallel -a subdomains.txt --joblog joblog --progress --bar
	"dig +noall +answer {}.<target domain>"

- Format of subdomains.txt:

foo
bar
admin

Dig with GNU Parallel (cheap method) vs Gobuster (proper & optimized tool) #

• Gobuster: Directory/file & DNS busting tool written in Go

$ git clone https://github.com/OJ/gobuster.git && cd gobuster
$ time go run main.go -u google.com -w ~/subdomains.txt -m dns -q -o google.com_subdomains_by_gobuster.txt

=> ~22s

• Dig: Native tool for DNS enumeration

$time parallel -a ~/subdomains.txt "dig +noall +answer {}.google.com | tee -a google.com_subdomains_by_parallel.txt"

=> ~27s

• Dig with GNU Parallel is almost as perfomant as Gobuster
• Very useful when you cannot install any tool on your attack machine (for example if provided by the client). You only have to install GNU Parallels

High-level scripting languages for easier static & dynamic analysis: Jython & Frida #

• Pentest sometimes involves reversing “custom-wtf” obfuscation or encryption
• Usually during Android applications & Java thick clients engagements
• Why use a high-level scripting language for instrumentation?
  • If you don’t want to go down the rabbit hole to figure out how it works
  • If you can’t replicate/rip the code into your favourite language
    • E.g.: Oracle WebLogic Server encrypts local passwords with a cryptosystem that’s not implemented in any Python module
• Use Jython for static analysis of Android & Java apps
  • Writing Java code in Python… that can use Java classes… and Python libraries in the same snippet
• Use Frida for everything else
  • Writing Python or JS or QML or Swift or .NET… injecting C++ scripted in JS (Google v8)… to instrument ASM, Objective-C or Dalvik… on Windows, Mac, Linux, Android, iOS
• Example: WebLogic Server password decryptor in Jython

Compile Python scripts on-the-fly: PyInstall #

• Pentest sometimes involves the need to have compiled version of tools
• Because:
  • The target you are onto does not have the proper execution environment (dependencies, interpreter) & you can’t install it (no root, no outgoing connection, laziness…)
  • You can’t just have a proper reverse shell or Meterpreter
  • You need to evade antivirus
• Compile Python tools with PyInstall
• PyInstaller bundles a Python script with a Python interpreter
• You can compile a script for Windows on Windows
  • Installing on Windows:
    • Install one “Visual C++ Compiler for Python”
    • $ pip install pyinstaller
    • This dependency is already satistied if you have Visual Studio installed
• You can also cross-compile for Windows from Linux with wine (Tutorial)
• Useful options:
  • --onefile: creates a standalone executable file which is a self-extracting zip payload
  • --onedir: creates a single directory with everything inside if you don’t want a standalone executable file, as large standalones (> 18 MB) take time to unzip before execution
  • --key <key>: a specific key to encrypt the zip payload, of course included in the executable (Tutorial)
    • Encrypt the payload & include the key in the executable (no obfuscation used), useful for anti-virus bypass
  • --icon <icon_file>: for visual fanciness

Examples of Python scripts compiled with PyInstaller #

• Impacket examples
  • Impacket example scripts compiled for Windows
  • Practical Usage of NTLM Hashes
  • High-value utilities in the Impacket tool suite: mimikatz.exe, ntlmrelay.exe, psexec.exe, samrdump.exe, secretsdump.exe, smbexec.exe, smbrelay.exe, ticketer.exe, wmiexec.exe, wmipersist.exe, wmiquery.exe
  • Real-life tricks
    • Impacket tools are gaining popularity and getting flagged by AV. Just use the --key option to bypass them!
    • If you cannot grab the Ntds database (Ntds.dit) of a domain because of the network connection or not enough space on your computer, just take the secretsdump.exe executable & put it on the target or on the domain controller wherever you want & you’ll be able to process the Ntds file remotely & grab password hashes from it
• Patator compiled for Windows
• CrackMapExec compiled for Windows
  • Thomas is the unofficial official maintainer of the compiled version 2
  • Version 2 is old but is UTF-8 compatible (useful for pentesting sites containing accents like French sites)
• jdwp-shellifier compiled for Windows

Demo #

• Compile rpivot from Linux for Linux

git clone https://github.com/klsecservices/rpivot.git
cd pivot
pyinstaller --clean --onefile server.py
./dist/server -h

Searching code: Stop using grep #

• Alternatives to grep:
  • sift
  • ripgrep
  • Silver Searcher
• They have more options & better performances

Taking a step back #

Conclusion:

• CSVKit
• Dataiku
• GNU Parallel
• Jython
• Frida
• PyInstall
  … all the things!

See you next time!