Web Enumeration Methodology

Photo by John Schnobrich on Unsplash

This article will describe the basics of Web Enumerations and is aimed at Beginners. It starts with basics and then gradually build up to more advanced techniques.

Table of Contents

-Introduction
-Enumerate and Make Notes
-Find out the Ports
-View Page Source
-Well Known Files
-Virtual Hosts
-Web Directories Busting
-Subdomains
-Web Technologies used
-Usernames
-Brute Force Login Pages
-SQL Injection in Login Pages
-Local File Inclusion(LFI)
-Cookies Manipulation and De-serialization Vulnerabilities

Introduction

Pen Testing is all about a structured approach towards a target. You can call it having a Methodology without which it is just a guessing game and trying things randomly, doing that might give you success initially but in the long run you must have an approach.

There are many such Methodology/Checklists out there for Web Enumeration. This is my attempt of creating a Beginner’s friendly one. This is inspired by doing many rooms on this amazing platform called TryHackMe and doing reading around in general about Web Enumeration.

My plan is to keep adding to this list about the new methods which I found and keep refining the approach toward Web Enumeration. Also this might come in handy if you are preparing for OSCP or doing CTFs. So lets start:

Enumerate and Make Notes

Before we dive in to technical stuff, this step is very important. Enumerate means to name or list things one by one. We should take notes at all the steps and if after doing all the steps we did not find anything, go check again all the notes if we missed something. In the end if nothing works take outside help and add the missed step to your Methodology. So you see you are already improving and this is an Iterative Process.

Find out the Ports

Websites usually run on Port 80 and 443 and other common ports are 8000,8080,8443. But they can run on any port, so use NMAP to find out where. Use the following commands to find out:

# Identify the list of services running on the target machine
⇒ sudo nmap -sS -Pn -T4 -p- TARGET_IP

Use the open ports found above and use in the next command to get more details about the open ports.

# Perform further information gathering on the open ports identified above
⇒ sudo nmap -O -A -Pn -T4 -p80,443 TARGET_IP

Browse to all the websites found above using TARGET_IP:PORT using your preferred browser. This will make sure that we don’t miss websites running on non standard ports.

View Page Source

This sounds simple but just right click and view page source of the Web Pages because they might contain:

  • Hints in case of CTFs.
  • Useful comments left out by the developers.
  • Links to application JavaScript files,endpoints which themselves can contains lot of useful information.

Well Known Files

Check for well known files which you can access via the browser, they might exist on the target:

  • robots.txt File: Access the file like http://TARGET_IP/robots.txt. This gives some of the end points which are basically excluded by web crawlers. Read more about robots.txt here. Try accessing these new end points found and see if that gives us some more information. Also keep checking the page source of the new end points found.
  • .well-know: This endpoints can contain lot of other details. Read more about this here. One such example is security.txt which is a proposed standard for websites’ security information that is meant to allow security researchers to easily report security vulnerabilities.
  • .git: Few websites can accidental expose their source code via this endpoint. If you find this then there is a P1 issue in real world as exposing a proprietary source code via this way is not acceptable and can expose a lot of juicy information.

Virtual Hosts

Sometimes visiting the target IP Address doesn’t reveal anything. Adding an entry in the /etc/hosts with the hostname and the IP address will help e.g.:

10.10.10.10    target1.thm
10.10.10.11 target2.htb

Try to access the target now like http://target1.thm. This is an example for rooms from TryHackMe. Depending upon the targets the entries will change. This works because of Virtual Hosting. Read more here and here.

Web Directories Busting

While browsing the target website you can find some files and directories, but to find more hidden stuff which are not that apparent to the user we can use Directory Busting tools like:

I have included few links in the above list which are extremely useful in exploring these tools. Here are few of the example which I usually use while doing Web Directory busting:

Dirb Examples

⇒ dirb http://sitename

⇒ dirb http://sitename /usr/share/dirb/wordlists/vulns/apache.txt

⇒ dirb http://sitename /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt

⇒ dirb http://sitename /usr/share/wordlists/dirb/common.txt

Dirsearch Examples

/**Basic Search**/
⇒ python3 dirsearch.py -u http://sitename -E -x 400,500 -r -t 100 -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt
/**Recursive Search**/
⇒ python3 dirsearch.py -E -x 400,500 -t 100 -u http://sitename -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -r -R 2

/**Search for custom extension**/
⇒ python3 dirsearch.py -e ticket -f -u http://sitename -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt
Where:
-E=default extension list
-r=recursive
-R= recursive level default is 1
-u=url
-f=force extension
-x=exclude status codes

GoBuster Example

/**Basic Search**/
⇒ gobuster dir -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -u http://sitename -t 40
/**Custom File extension Search**/
⇒ gobuster dir -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -u http://sitename -t 40 -x xxa
/**Dump results to file**/
⇒ gobuster dir -u http://sitename -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -o gobuster.txt
/**File extension Search, Big list**/
⇒ gobuster dir -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -x .php,.asp,.aspx,.jsp,.js,.do,.action,.html,.json,.yml,.yaml,.xml,.cfg,.bak,.txt,.md,.sql,.zip,.tar.gz,.tgz -u http://sitename/secret -t 40
/**File extension Search, Big list, with positive status codes**/
⇒ gobuster dir -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -x .php,.asp,.aspx,.jsp,.js,.do,.action,.html,.json,.yml,.yaml,.xml,.cfg,.bak,.txt,.md,.sql,.zip,.tar.gz,.tgz -u 10.10.141.98/secret -t 40 -s 200

ffuf Examples

/**Basic Search**/
⇒ ffuf -u http://sitename/FUZZ -w /usr/share/seclists/Discovery/Web-Content/big.txt

/**With Recursion**/
⇒ ffuf -u http://sitename/FUZZ -w /usr/share/seclists/Discovery/Web-Content/big.txt -recursion

/**With File Extensions**/
⇒ ffuf -u http://sitename/FUZZ -w /usr/share/seclists/Discovery/Web-Content/big.txt -recursion -e .bak,.aspx,.html,.php,.txt

/**Basic Authorisation**/
⇒ ffuf -u http://sitename/FUZZ -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -c -H "Authorization: Basic cmFzY2FsOmJyaXR0YW55" -s

There is no right or wrong way to approach this. You can use any tool which you prefer. I suggest try them from top to bottom from the list above and choose which one suits you best. My preferred one is ffuf.

Subdomains

Subdomains is a domain which is a part of another domain which is usually the main domain. If a target website has a subdomain then it increase our chances to gain our initial footholds on the target. There are many tools which can help us in finding/Brute-Forcing sub domains. Here are two of them:

  • WFuzz: Web application fuzzer.

Here is an example from one of the TryHackMe’s room:

wfuzz -c -f subdomains.txt -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt -u “http://cmess.thm/" -H “Host: FUZZ.cmess.thm” --hl 107

Here is the Output of the above command:

wfuzz

So we have found one sub domain, add it to /etc/hosts file as shown below and access it like: http://dev.cmess.thm

  • fuff: Fast web fuzzer written in Go

For example:

ffuf -u “http://FUZZ.cmess.thm" -w /usr/share/seclists/Discovery/DNS/subdomains-top1million-5000.txt -v

Web Technologies used

It is useful to know what technologies our target Website is using. Identify then using the following tools:

Sample Output for Whatweb:

whatweb example

Sample output for the same website using the browser extension wappalyzer:

wappalyzer

So from the output of both the tools you can see the technologies used for this site and even versions for few of them. Now it is possible that there are know exploits for some of the technologies which were identified above for e.g. Apache 2.4.7 might be vulnerable and there can a public exploit available. Use tools like searchsploit if you are using kali as your attacking machine to find vulnerabilities for this version or search in the interwebs to find one. Also few other source are:

If you find one don’t jump to exploiting it, note it down and carry on enumerating.

Usernames

While browsing the websites and the source pages keep an eye on the names which you see like in comments, articles etc and keep a note of them.

Brute Force Login Pages

While browsing through the target if you come across any kind of login page, you might try Brute Forcing the login page. There is a very good list from the SecLists project for User Names. Add your own user names discovered so far on the top of the list which you have chosen. Hydra is a very good tool to do this job. Little investigation is needed to format the hydra command depending upon how the login page is setup. One such example is a http post form request:

hydra                      \
-L /path/to/usernames \
-P /path/to/password/list \
TARGET_IP \
http-post-form \
"/login/authenticate.php:username=^USER^&password=^PASS^&submit=Login:F=Incorrect" -V

This looks scary at first, but it is relatively simple. View the source of the login page, for this particular example:

hydra login brute force example

First 4 parameters are self explanatory. The last one ties with the login form. We need to give full path to the login page in this case it was /login/authenticate.php. Match the names for username and password with there respective capitalized parameters and the submit button with the value Login. The ‘F’ is the text which the webapp return when wrong values are put in and ‘V’ is for verbosity. Use password list rockyou or any other useful password list from SecLists project.

If the WebApp supports Basic Authentication use the following command:

hydra -L /usr/share/seclists/Usernames/top-usernames-shortlist.txt -P /usr/share/wordlists/rockyou.txt TARGET_IP http-get

Here are few more examples from various other scenarios:

⇒ hydra -l kwheel -P /usr/share/wordlists/rockyou.txt $IP http-post-form "/wp-login.php:log=^USER^&pwd=^PASS^&wp-submit=Log+In&redirect_to=http%3A%2F%2Fblog.thm%2Fwp-admin%2F&testcookie=1:F=The password you entered for the username" -V

⇒ hydra -l user1 -P /usr/share/wordlists/rockyou.txt $IP http-post-form "/wp-login.php:log=^USER^&pwd=^PASS^&wp-submit=Log In&redirect_to=http%3A%2F%2F10.10.44.249%2Fwp-admin%2F&testcookie=1:F=ERROR"

⇒ hydra -l user1 -P /usr/share/wordlists/rockyou.txt $IP http-post-form "/simple/admin/login.php:username=^USER^&password=^PASS^&loginsubmit=Submit:F=incorrect"

⇒ hydra -l user1 -P /usr/share/wordlists/rockyou.txt $IP http-post-form "/login:username=^USER^&password=^PASS^:F=incorrect" -V

⇒ hydra -l admin -P /usr/share/wordlists/rockyou.txt $IP http-post-form "/admin/index.php:user=^USER^&pass=^PASS^:F=invalid"

Please be mindful that of doing any brute forcing on a live target is dangerous thus use this option carefully. For CTFs and Practice rooms mostly this is ok.

SQL Injection in Login Pages

It might be possible to bypass the login pages all together if they are vulnerable to SQLi. Try the following in the Login Name field and see what you get:

⇒ SQL Injection 101, Login tricks
admin' --
admin' #
admin'/*
' or 1=1--
' or 1=1#
' or 1=1/*
') or '1'='1--
') or ('1'='1--

If you are lucky you might get in. Well this process can be automated using Burp Suite. Set the Burp Suite up, capture the login request in Burp Proxy and send that to Intruder Tab and add the new payload marker on the Username. Paste a SQLi payload list in the Payload tab->Payload Options[Simple list]. Keep the Attack Type as Sniper and start the attack. In the output observe the Length column to see if you find a record having a different value then the others, that might be the payload which was successful.

Some awesome resources for bypassing Login using SQLi are:

Also here are some great lists for SQLi payloads:

Local File Inclusion(LFI)

LFI lets attackers read sensitive files from the target. The basic reason for this vulnerability is that the unsanitized user input is processed by the web application,resulting in sensitive files exposures.

Read my following write-ups which shows how to exploit LFI.

Basic: https://0xsanz.medium.com/lfi-inclusion-tryhackme-d66943f7ae4c

Using php filters : https://0xsanz.medium.com/archangel-tryhackme-436ca1127513

LFI To RCE: https://0xsanz.medium.com/team-tryhackme-2a9842ac5eed

There are few ways to get Reverse Code Execution from LFI. They are:

  • Using PHP filters and Input
  • Apache Log Poisoning
  • SSH Log Poisoning
  • Proc Self Environ
  • Proc Self Fd

Some of them are covered above write-ups.

Cookies Manipulation and De-serialization Vulnerabilities

Many websites use Cookies to store and validate users. Cookies can be exploited in many ways if the Web Application doesn’t handle them properly. Capture the request in Burp Suite which may be related to Login, Sessions, Shopping Carts etc .Few things that can be exploited are:

  • Change a username with admin username and submit the request again. May be doing that will allow you to login as admin user. Try same things with different username may be one user has got more privileges than the other.
  • If there is some flag in the cookie say for e.g. IsAdmin=false , then change it to IdAdmin=true and resend the request and check if that gets you admin access.

So this is just an introduction to things which you should try while doing Web App Pen-testing. Obviously this is a huge topic and there are many more vulnerabilities which exists and can be exploited, but these techniques will definitely help you in OSCP preparation, CTFs and in doing Basic Web Application testing. Use this as a spring board to explore more in this area.

If you liked this article then please share, clap and follow me. I would love to see your comments on how this can be improved further. Thanks!

--

--

--

Software Developer having keen interest in Security, Privacy and Pen-testing. Certs:- Security+,PenTest+,AZ900

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

RimWorld — My Electric Goblin Wall-lamp

Concurrency is not just a Computer Science problem

Run Jenkins in a Docker container — part 3 — run as root user

Simple Player Movement in Unity

The 5 Best Visual Studio Code Extensions

computer

App Store & Google Play: Submission Checklist

Simulating A Real-world SQL Environment

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
0xsanz

0xsanz

Software Developer having keen interest in Security, Privacy and Pen-testing. Certs:- Security+,PenTest+,AZ900

More from Medium

Try Hack Me: Cyborg

CyberHeroes | TryHackMe | WriteUp | MZS

Meet The Prince0f4llSaiyanz

Pandora — HTB Walkthrough