Posted in Podcast on March 18, 2019
The Bug Hunter Podcast 5: Recon workflow & Out of the box thinking in day-to-day life
Posted in Podcast on April 1, 2019
Hi, here’s a new episode of the Bug Hunter podcast!
If your favorite podcasting app is missing from this list, please let me know so I can add it.
Also, if you prefer written text, you’ll find the whole transcript below. It’s also helpful for finding any links or commands mentioned in the audio.
Hey hackers! This is the Bug Hunter podcast by Pentesterland. The podcast for pentesters & bug bounty hunters.
We tackle technical questions & inspirational topics to help you develop both a hacker skillset & mindset.
Welcome to episode 5! I’m you host, Mariem. And the title of this episode is: “Recon workflow & Out of the box thinking in day-to-day life”.
The two segments I have today are:
- Q&A: on how to create your own recon customized workflow
- Personal development: on thinking out of the box in life in general
As usual, half of this episode is not technical. It’s about cultivating a different mindset that will help you in life and thus in hacking too.
Also, the transcript for this episode is on pentester.land/categories/podcast/. Under episode n°5, you’ll find this show’s transcript, including any links and tools I mention.
Some time ago, I sent an email to people subscribed to pentester.land asking them for help. I was in the process of creating this podcast and needed to know which issues related to pentesting & bug bounty my readers struggled with to address relevant topics.
One of the questions I got is from Ján Masarik. He said:
I would really appreciate insight into specific workflow of best bugbounty hunters. Something specific, eg.:
- Start recon tool on scope
- Use the product for 30min (jobert)
- Gather JS from crawl and spider
- Gather all s3 buckets and start enumeration ….
Not sure if that is possible, but it would be awesome if you could find someone willing to share it.
Thanks again and Merry Christmas!
Yep, it dates back to Christmas. But I try to answer everyone even it takes some time due to an overloaded calendar. And I think the question of hacking workflow is timeless.
So thank you Ján for this question! I appreciate that you took the time to formulate and share it.
Before answering you, I want to talk a little bit about workflows, what they are and why they’re so useful for Web app testing. Then we’ll get into why you won’t find a perfect one that’s ready to use, where to find information on what successful bug hunters are doing and how you can create your own workflow.
Whether you do your recon and testing manually or with automation, you need to have a base workflow to follow. The reason is that there are so many tools, tests and sources of information… If you’re not organized, you will run tools and do tests randomly, forgetting some of them. And you will eventually miss something important that would have given you a bug.
But I’m not talking here about a checklist.
A workflow is like an algorithm. It starts with an input, like the target’s name or domain. You run some tools or tests to get a specific piece of information. Then you take the results and feed them to the next tool or test.
For example, you run a set of tools to get subdomains, then you pass the subdomains list to another set of tools to test for thinks like CORS misconfigurations, directory bruteforce, subdomain takeover, etc.
If you want to see what this looks like in practice, head to pentester.land/cheatsheets. The entry “Compilation of recon workflows” is a collection of recon flowcharts I found online. It will give you an idea of what you need to create for yourself.
You can see that this is different from checklists or mindmaps.
For the little story, I created a custom recon tool a while ago. It automates repetitive steps like domains and subdomains enumeration, getting DNS information, port scanning, taking screenshots, testing for some vulnerabilities, and more.
The tool did a lot of different things, and it had 3 different modes: fully automatic, semi-automatic and manual. So I used a mindmap to visualize all the different steps and tools it called.
But the mindmap format didn’t work well for me. The first version of my tool was way too complicated, tried to do too many things from the start, and the mindmap didn’t help clear my thoughts. I ended up adding links and notes all over the place.
The workflow format was perfect for simplifying my testing process. I used a dedicated software that doesn’t allow for taking notes or adding links. All you can do with it is draw a workflow like the ones on my “Compilation of recon workflows”.
This forced me to re-think my testing process, simplify it, and make sure to understand what kinds of input a tool needs and all the different outputs it can give. This was a great first step for better automation of my recon.
So I highly encourage you to draw a flowchart of what you are doing right now when you’re testing Web apps. It is the best way to get clarity on what is missing, what can be improved and how to automate it all.
So, now that you’re sold on workflows, where can you find one?
I’m convinced that no one will share their complete recon or testing workflow, unless maybe you’re BFFs or if you’re paying them a ton of money. Honestly, I didn’t find one really complete recon or testing workflow out there. The reason is simple: if a hacker tells everyone what they’re doing to differentiate themselves, they will lose the edge.
People like @NahamSec and @JHaddix are very generous and share a lot about their recon process. But I think that many techniques are still missing from their methodology either because they truly don’t want to use them, or because they didn’t have the time to evaluate them, or because new techniques and tools are published all the time. So the methodology must be often updated. So why would you just copy someone else’s process and maybe deprive yourself from something interesting that could help you find bugs?
As a bug hunter or pentester, you want to copy what successful hackers are doing, but take it as your bare minimum. Then try to find new ways to distinguish yourself like new techniques that no one is talking about or that you have to dig deep into an interview to hear a small hint about. It’s what will allow you to find unique results, whether it’s new endpoints or bugs that others are missing because they’re all doing the same kind of recon process.
So Ján, I’m not interviewing any famous bug hunter for this podcast. But even if I did, they would only give you some pieces of their workflow. And you would still need to do the additional work of reading everything you get your hands on to complete that workflow… any bug writeups, tutorials, AMAs, tweets, etc, are sources of information that are already public and can inspire you to add tests and tools to your methodology.
Compared to other industries, there is already a lot of free valuable information shared accross Twitter and blog posts on recon, automation, and testing methodologies.
Things change so fast in this field. New tools are released all the time, that outperform the previous tools or introduce new techniques.
New conference talks and interviews also present new ideas that you want to incorporate as quickly as possible to your flow.
So it’s a good idea to get into the habit of consuming all the information that you find out there and build your own customized unique workflow.
That’s why I didn’t contact any hackers for their insight on this, like I did for the last podcast.
What I suggest is:
- Pick a tool for drawing diagrams. The one I use is Dia. Another good one is draw.io. There are may other tools both online & offline. Just pick one.
- Draw a flowchart of what you’re currently doing. What do you usually start with, a domain, an organization name? What do you do next? Run Amass? Then massdsns? Nmap?… Include all the tests and tools you use.
- Look at what other people are doing & Improve your own workflow.
- Rinse & repeat (meaning, continue tracking what others are doing & improving your workflow)
This is my answer to you Ján. If I do interviews or AMAs in the future, I will still try to convey your question to hackers I interview.
But for now, my advice for you is that your question is already answered. The answer is just scattered accross the Internet. Very few hackers share their whole process, but a lot of them share bits and pieces that you have to pay attention to find.
You have to organize your notes (drawing your workflow is a great way for that), and pay attention to what’s already shared out there. Pay attention to the details, consume existing interviews, AMAs, podcasts, blog posts, tweets, and you will get the information you need and maybe even new ideas.
To help you get started, I will share a recon cheatsheet on the blog next week. It will be a list of articles, tools and references for recon. Basically a lot of information on what other bug hunters are doing.
Until then, start your homework. Draw a flowchart of your recon and testing process.
Here is a tip for that:
Use different geometric shapes in your diagram. For example, if something is an information (like subdomains, IPs, or open ports), use a parallelogram. If it’s an operation (like running a tool), use a rectangle. If it’s a test, use a different shape too.
This will make the flowchart easier to read.
Also, here is an example of a set of recon steps based on what I’ve seen shared by other hackers & what I do myself. It’s not complete but this is the bare minimum that you can do:
- Enumerate domains
- Enumerate subdomains
- Resolve subdomains
- Test for CORS
- Test for subdomain takeover
- Scan for open ports
- Make a list of Web apps
- Do network tests on non HTTP ports
- Take screenshots for visual identification
- Run Web app scanners on the list of Web apps
- Run CMS scanners
- Do files & directory bruteforce
- Enumerate JS files
- Get secrets & new endpoints from JS files
- Look for known vulnerabilities based on identified service & library versions
- Do parameter bruteforcing
- Github recon
- Test for AWS misconfigurations
- Google dorking
A lot of bug hunters automate most of these steps. And while their custom tool is running on a target, they manually browse the app, use all features, get to undertand them then start testing for vulnerabilities.
But that’s not how everyone operates. Every hacker is unique. Some prefer to focus on one feature or on type of vulnerability at a time. Others look for any kind of bug: They do a first pass to understand it and find low hanging fruits, a second and third pass to find less obvious bugs.
So read everything you can, but your true answer on what your recon and testing process will look like will only come from you actually testing apps and seeing what works for you.
I hope this segment helped get a better grasp of this whole workflow thing.
Keep an eye on pentester.land for next week’s recon cheatsheet. Between that and The 5 Hacking NewsLetter, you have more than enough information to build your personalized testing workflow.
FIY, I will start doing one podcast every two weeks, instead of every week. Burnout has snuck up on me again, so I have to slow down.
If you have any questions or suggestions for this podcast, send us a DM or an email.
I’m always happy to hear from you and answer your questions, even if it takes time sometimes!
Personal development: Developping out of the box thinking in day-to-day life
I’m a rather atypical person. Wherever I go, I stand out, not for the sake of being different, but because I am not actively trying to fit in. I’ve never wanted to fit in even as a teenager. Social pressure facilitated by the need to fit in, is what leads a lot of people to do things they don’t believe in just to avoid standing out, to avoid being the one who’s different than everyone else.
That’s how some very young people start smoking or drinking. That’s why some students get into professions they dislike, to satisfy their parent’s view of what is a socially acceptable job.
Of course, there is no absolute right or wrong in this life. Sometimes, doing a job you’re not passionate about is really your only option to survive. Sometimes, following your passion like full-time bug hunting isn’t a good idea or it is, but it’s just not the right timing.
But only you get to decide, no one else. Others can and should advise you but not impose their choices on you.
I truly believe that the only way to be happy, content with your life, and have inner peace is if you develop independent thinking.
There are important questions that define your life, your ultimate goals, your day-to-day job, hobbies, life, routine. These are questions that define how you interact with other people, if you love yourself, if you will have any regrets when you are dying… These questions must be formed and answered by you and only you.
Despite social pressure, especially the pressure that family and parents put on you… You are the master of your own life. You are the own who makes the decisions. Do not let fear of disappointing or shocking others define your questions & answers to life… because a lot of times, people are shocked or offended at first and then get used to change, especially if they see that you are happier or healthier.
This topic is very dear to my heart, because I come from a conservative family. Change and different thinking weren’t encouraged. Innovative ideas and principles were often considered harmful. Being different felt like a failure.
And this is wrong, because your difference is what makes unique. It’s what gives you the power to do things differently and maybe change the world, produce something that no one else could have done.
So even though I was always questionning things, I ended up keeping it to myself. For years, I wanted to make so many changes but didn’t in order to avoid sturring any trouble.
So here are 6 ideas that I’ve been implementing these last few years. Next week, I will give 6 more tips on this same topic. I prefer to split this segment across two episodes to avoid overloading you with information.
These tips or ideas have helped me a lot develop independent thinking and face any resistance I got:
Challenge yourself. Push your limits. You are capable of so much! More than you realize now… A lot of times, people undermine themselves by thinking too small. You will hear things like: “I’m too old”, “I can’t learn this and change careers now, if only I was younger”, “I’m not smart enough”, or “This is never going to work”… Sometimes, you don’t think this but other people will tell you: “You’re not smart enough”, “What?! YOU, creating a new business?”, “You’re not a good pentester, you can’t find squat”, “You can’t do bug bounty, just leave it alone and do a real job”… This kind of small thinking can lead to burying any hope of change, defying odds, and undertaking unique visionary endeavors.
Steve Jobs was a great example of an out of the box thinker. Here is what Apple once said:
The crazy ones, the misfits, the rebels, the troublemakers. The round pegs in the square holes. The ones who see things differently. They’re not fond of rules and have no respect for the status quo. You can quote them, disagree with them, glorify or vilify them. About the only thing you can’t do is ignore them, because they change things. They push the human race forward. And while some may see them as the crazy ones, we see genius. Because the people who are crazy enough to think they can change the world, are the ones who do.
I agree with this 1000%. I am a firm believer that everything can be changed. Every shortcoming can be worked on and improved if you truly want it. Any one of us has the potential to change the world. There is no place for self-limiting beliefs.
This is important to internalize. Do not let people or your pessimistic inner voice dictate what you can and can’t do. Challenge yourself, think BIG. Even if you don’t know now how you will achieve a goal, it doesn’t mean that you can’t try. Define your unrealistic completely crazy goal, and you will figure things out as you go, one after the other.
Thinking goes hand in hand with action
It is not enough to use your head, you have to act. This applies to both life and hacking. The reason is that by acting on a new idea, you get feedback, real-life experience, which is the only way to review your idea, improve it and advance.
For instance, let’s say you read about healthy eating. You form the conviction that junk food is bad for you and you want to change your eating habits. But you don’t know where to start because there are so many conflicting opinions amongst doctors and also in your own family. The best way in my opinion is to do some research or consult a professional, choose something you want to try like plant-based eating or paleo and do it for a few weeks. The experience you get will help you understand how your body reacts, and figure out what works for you. You can’t arrive to the same conclusions and find your truth just by reading what others say about it. Also, you will always meet people who will try to discourage you from the path you chose. So the only way to distiguish good from bad ideas is by testing and figuring it out yourself.
This is a virtuous circle. New ideas must be followed by action. And action provides the experience to review your ideas and improve them.
It gets easier with practice
Thinking is an activity that gets easier and easier with practice. And not everyone does it, believe me! Intellectual laziness is a thing. That’s why whole communities end up with weird ridiculous beliefs.
Out of the box thinking applies to everything, not only hacking: Do you practice your religion because your parents did the same way, or because you truly believe in it? Do you eat the way you do because it’s how everyone around you is eating, or because it’s what is best for your body according to science and your own experience? Are you living your life according to your own definition of success or are you chasing someone else’s dreams?
Go live abroad
Living alone for some time and traveling can really help you get to the bottom of who you are and what makes you happy or not, which kind of person you want to be. It can help you arrive to some conclusions that are not mainstream and might be considered out of the box, different. Being surrounded by people who resist your change or your different aspirations can deter you from following them and keep you living inside that normal common bubble where most people live. Traveling and going away on your own personal adventures, even for a few months, can help tremendously. Had I not lived abroad as a student, I would have never gone off meat and sugar, beat boulimia, and I wouldn’t have found my own answers to important questions like which religion I believe in, which principles I want to live by, and the kind of person I want to be.
Get exposed to what others are doing
Traveling abroad (even as a tourist for shorts period of time) makes you discover new different cultures and realize that it’s OK to be different. Being vegetarian in Morocco is really weird and discouraged, but it’s common in big parts of India. Reading about other people’s life hacks, life philosophies, and creative solutions to common problem will help you broaden you horizon. It even makes you smarter according to research.
You’ll find in the show’s transcript a link to a BBC article mentioning new research on flexible thinking. This paragraph from the article is a great summary of the findings:
When people are exposed to a more diverse group of people, their brains are forced to process complex and unexpected information. The more people do this, the better they become at producing complex and unexpected information themselves. This trains us to more readily look beyond the obvious - precisely the hallmark of creative thinking.
This applies to hacking too: the more writeups, articles & research your read, the more ideas you will have. Many times hackers read about an existing attack, and adapt it to create a variant. For example, @vulnano found a $10,000 bug on Facebook by uploading a corrupted GIF file which caused memory disclosure. He got the idea from gifoeb, a tool for exploiting a similar ImageMagick bug.
Musicians do this too, all the time. I was listening to a podcast about creative thinking. It’s The art of manliness Ep. 432. How to achieve creative success. It explains that creators, especially musicians, are the biggest consumers of culture. And imitation is a huge part of their creative process.
Do your own research
It is easier today thanks to the Internet. We have access to so much valuable information, research, reviews and feedbacks from literally millions of people on any topic or product.
So do your own research, think, experiment and don’t pay attention to what others think.
I’ve read a lot about healthy living and eating, and I made radical changes that were unheard of where I live. I got a lot of resistance from my close family. But I didn’t care because they have been brain washed by all the lobbies, industrial giants and doctors that don’t know what they’re talking about or are only after our money. And it’s only now that the same people are starting to realize that they were lied to about pretty much everything: milk, sugar, meat, flour, organic vs non organic food, drugs…
If your thoughts are backed by science or common sense, don’t let anyone convince you to do otherwise.
Before wrapping up, I want to share a cartoon I’ve seen on Twitter. It’s the ultimate answer to the question: “Where do I start…” / “What should I study….”
It’s better seen than explained, so just open this podcast’s transcript on the blog and you’ll find it:
For some reason, it makes me laugh every time I see it. And the next time one of you asks me how to start something, this will be my response…! :)
That’s it for today guys!
I hope you found this episode unteresting. I would love your feedback.
If you did, stay tuned in for more tips on how to think out of the box in the next episode. And send us any question that you have either by DM on Twitter at twitter.com/pentesterland or send us an email to [email protected].
Thanks for listening to The Bug Hunter podcast. Please share with your friends and colleagues, like, subscribe and comment.
See you next time! Keep on hacking!
If you enjoyed reading this, please consider sharing it, leaving a comment, suggestions, questions…