In our last post Redefining Assets – A Modern Perspective we talked about how the definition of an ASSET has evolved with time and is now more inclusive.
In this post, we are going to talk about why and how asset discovery should be done while manually testing the web application. We are also releasing a Burp Suite extension to automate this process.
Extension Name: Asset Discover
Platform: Jython
Download Link: https://github.com/redhuntlabs/BurpSuite-Asset_Discover
Scanner Mode: Passive
Why?
The outcome of any security assessment program, be it vulnerability assessment, penetration test, or red team is limited by its scope. We cannot remediate the risks associated with a particular resource, which is not part of the scope of the assessment, as they are simply unknown unknowns. The answer to this problem is keeping a track of assets using continuous asset discovery.
How?
The asset discovery process is simple, but not easy. We need to start with some seed information about the organization such as company name, domain(s), IP(s) etc.; have a list of data sources mapped with seed type and then extract information from them. Also, this process needs to be recursive in nature to check if the newly identified information could become the seed information for another source. Our co-founder Shubham, talked about it briefly during his OSINT for Proactive Defense – RootConf 2019 presentation.
One primary challenge with asset discovery is the trust level on the authenticity and relevance of the information identified. Any information source which is directly owned/managed by the organization have a high trust level, on the other hand, third-party source might require multiple checks to validate the information. One such source which can be relied upon is the websites owned by the target organization. Ofcourse, further validation must be done on the assets identified from this source yet we can have a significant level of trust on them.
Some of the assets we can find from a website are domains, subdomains, third-party services, public and private IP addresses, cloud storage, tracking codes etc.
The simplest way to do this is to look for specific patterns in the response of website pages.
However, doing this for one or more websites would be very tedious. To automate this, we either need to write/import a crawler which parses all responses for regular expressions or create an addon for existing tools. As Burp Suite is the goto tool for most of the people dealing with web security, we wrote a Burp Suite extension ‘Asset Discover’ for this.
How it works?
The extension acts as a passive scanner which parses the response of the pages that are in scope and constantly keep an eye for an asset. These assets are identified and classified based on RegEx patterns for different kinds of assets.
Setup
To use the extension, start the BurpSuite application and setup the python environment by providing the Jython.jar file in the Options tab under Extender.
Now perform the following steps:
- Download the extension.
- In the Extensions tab under Extender, select Add
- Change the extension type to Python
- Provide the path of the file Asset_Discover.py as shown in the figure below.
Once imported, the extension will be listed in the Burp Extensions.
Usage
Now the setup is complete. Let’s pick a target application and add the URL to the scope. While you do manual testing or browse through the website, the extension will use the passive scanner to sniff the web pages, identify the assets and list them under the Issues section.
The issues reported will appear as Asset Discovered: < Asset Type> with a nested list of identified assets in that category.
As shown in figure below, multiple assets could be easily identified using this extension.
For this first release we have added a limited number of signatures, however there are multiple future development ideas that can be worked upon, such as:
- adding signatures for more asset types
- consolidate the results for each URL
- improve the regular expressions
- handle errors/exceptions
- remove false positives etc.
Download Link:
https://github.com/redhuntlabs/BurpSuite-Asset_Discover
If you want a managed solution that can help you find and secure assets belonging to your organization and check their Attack surface in a continuous fashion, you can explore our SaaS offering NVADR.