July 17, 2020 0

Open Source Intelligence: What Is OSINT & How Does It Work?

in Cyber Security

(20 votes, average: 4.05 out of 5)

Loading...

Juniper Research estimates that there will be 83 billion IoT connections by 2024 — that’s 10 times more IoT devices that will be generating data than the estimated number of people who will be alive on Earth at that time

Wondering “what is OSINT?” The term, which stands for open source intelligence, refers to a system of gathering data from freely available public resources. (So, if you’re also wondering “what is open source information,” this article will answer that question, too.) But what makes the process of using open source intelligence different from general data collection is that it goes beyond querying search engines using different permutations of the same phrase.

OSINT is a term that originated with the U.S. military in the 1980s. They needed to find a way to keep up with dynamic information to keep a tactical frontline advantage. Nowadays, professionals across various industries use OSINT data to achieve different functions. For example, marketing and sales teams use it to increase conversions, whereas cybersecurity teams use it to conduct investigations and mitigate threats.

Today we’re going to discuss what OSINT is, cover some of the most popular OSINT tools and techniques, the OSINT framework, and much more. But before we dive into that, let’s start by defining open source intelligence and open source information.

What Is Open Source Intelligence (OSINT) & How Does It Relate to Open Source Information?

OSINT is a term that refers to a framework of processes, tools, and techniques for collecting data passively from open or publicly available resources (not to be confused with open-source software). Open source intelligence historically referred to open source information gathering via conventional channels such as newspapers, radio, TV, etc. Nowadays, to extract specific intelligence, we use:

• Blogs,
• Discussion boards,
• Social media,
• The dark web (accessible through TOR), and
• Deep web (pages not indexed by Google like a people search database).

In some cases, such as with social media, OSINT has developed into a prominent subset of its own called SOCMINT, which is short for “social media intelligence.”

Examples of open source intelligence gathering include:

• Searching for information about a competitor’s employees or services,
• Law enforcement agencies gathering intelligence using online public resources to prevent crimes,
• Identifying vulnerabilities to exploit at a later stage on a target system or network, and
• Collecting information to conduct a social engineering attack.

Advantages of Using OSINT

Open source intelligence gathering comes with several benefits. Let’s take a look at some of them:

• If you’re on a budget, conventional information collecting techniques and tools may not be an economically viable solution. One of the main benefits of employing OSINT to gather intelligence is that it involves a minimal level of financial investments.
• The information obtained is not classified and has been divulged freely, hence it is legal to obtain any such information.
• Because it relies on public resources, users frequently share and update the information regularly.
• Business owners and decision makers can gain perceptivity through OSINT data that can facilitate building long-term strategies for a variety of business goals.
• OSINT can also be an invaluable tool in matters of national security.

Disadvantages of OSINT

Now that we understand the convenience of using OSINT, what are its drawbacks? Like you’ve probably guessed, as easily as you can utilize OSINT to gather intelligence, an adversary can also use it to collect information about you or your business. Besides that, here’s a list of a few other disadvantages you may run into while using OSINT:

• Finding information means very little unless you can put it to use in some meaningful way. Filtering out junk data from valuable information can be challenging based on the volume of data you find.
• Once you have filtered out usable data, you need to validate that the information is reliable. Organizations and individuals may deliberately post false information to mislead potential attackers.
• The information gleaned is not consumable as it is, and there is a considerable amount of analysis work involved.

How OSINT Relates to Cybersecurity

In cybersecurity, OSINT techniques aren’t a one-size-fits-all approach. Depending on the purpose of your research, your end goal, and what you’re trying to find, the techniques you deploy will vary as will the tools used. Once you determine who your target is and the steps you’ll take to conduct your research, then you can choose the appropriate tool and approach.

While gathering INT data, the basic idea is to:

• Connect the dots,
• Pivot to a new source of information as needed to build a deeper intelligence profile, and
• Validate every assumption that’s made along the way.

If you’re conducting OSINT as a part of an engagement, generating a report at the end with screenshots attached is an indispensable part of the process.

OSINT is frequently used to profile a target, and it’s done by conducting passive reconnaissance to glean information without actively engaging with the individual or corporation. However, there are some obstacles to gathering intelligence. An account created specifically to conduct OSINT on, for example, Facebook, might end up looking like a fake account. Some websites actively delete such accounts that don’t look legitimate. Moreover, the sheer volume of available data that you need to analyze and process to gain useful insights can be overwhelming.

OSINT data is valuable in the later stages of an attack because it adds credibility if and when there’s direct contact with the target. It allows virtually anyone to craft a customized attack that exploits weaknesses in people, processes, or technologies.

Using OSINT as Counterintelligence in Organizations

Counterintelligence refers to activities concerned with detecting and neutralizing threats to an organization’s security against any opposition’s intelligence service. The first move any hacker makes when planning an attack is to gather as much information as possible. This can be information about the target organization, specific employees, and any other available data that’s useful. The next step is to take all of this information they gather from various sources and turn it into intelligence through analysis and correlation.

By utilizing OSINT capabilities, your organization can take steps to identify all publicly disclosed information. You can use this intelligence to scrub the data to prevent disclosure of sensitive information or to train your employees to be aware of it. Having a dedicated team that identifies correlatable data to form intelligence is invaluable. They can help you avoid reputational damage by identifying and then attempting to obscure or censor any publicly disclosed information of a sensitive nature where possible. Additionally, these insights may also prevent or buy you time to mitigate the risk of any potential data breach due to such exposure.

What is the OSINT Framework?

The OSINT framework provides a collection of OSINT tools, classified into various categories, that pentesters and hackers alike can use for reconnaissance. The OSINT framework has a web-based interface and is primarily focused on listing free resources.

For instance, the first entry “username” can be explored in our OSINT research if we focus on discovering usernames utilized by a target across various accounts on the internet. On clicking the entry, it’ll display a list of all the tools that can be employed to accomplish this goal.

6 OSINT Tools That Can Enhance Your Cyber Security Efforts

Scouring the internet manually to profile your target organization or individual can be time consuming. Thankfully, the new generation of “OSINT-ware” removes this obstacle for attackers and pentesters alike. These tools help them to quickly determine, with very little effort, the finer details about a target’s network.

DuckDuckGo, Google Maps, Pastebin, and social media sites are good places to start and are commonly used. However, there are several additional tools that might help you gather intelligence more efficiently:

1. Shodan

The first OSINT tool we’ll discuss is Shodan, which stands for Sentient Hyper Optimized Data Access Network. This search engine for interconnected devices allows you to search for IoT/SCADA devices, routers, traffic cameras, and more.

Shodan search attempts to grab data such as the service, software, version number, or other information from the ports it scans. The tool comes with filters such as country, port, operating system, product, version, hostnames, etc. that helps narrow down the results. It displays a vast amount of insecure information that’s freely available and access to web interfaces of IoT devices with weak or default passwords, devices like webcams at people’s homes, and other unsecured appliances.

Pentester can use Shodan to find insecure web services while conducting vulnerability assessments. The tool comes with a free plan that offers a limited number of scans, or you have the option of using a paid version. However, organizations can request to block Shodan from crawling their network.

2. Maltego

Maltego is an aggregator of interfaces to several OSINT databases and covers infrastructural reconnaissance. This tool can harvest a wealth of sensitive information about any target organization, such as:

• Email addresses of employees,
• Confidential files that have carelessly been made publicly accessible,
• DNS records, and
• IP address information.

You can also use Maltego for personal reconnaissance to collect individual-specific data. Maltego communicates with search engines on the internet to gather all this information in one convenient location.

3. Metagoofil

Metagoofil is another OSINT tool that utilizes the Google search engine to extract metadata from publicly available files (.pdf, .doc, .xls, .ppt, etc.) belonging to any target company. After downloading the documents onto the local disk, it pulls out the metadata using various libraries like Hachoir, PdfMiner, etc. and generates a report.

4. GHDB

Hacker and cybersecurity Expert Johnny Long developed the Google Hacking Database (GHDB) for pentesters in 2000. It’s a list of search queries that reveal interesting information that was, in all probability, made public unintentionally. An example of unintended disclosure could be a search engine crawling a web document that contains a link that holds sensitive data. The search engine may subsequently follow it and index any information on it.

Some of the information that you can query using GHDB includes verbose error messages that contain sensitive information like:

• Directory paths,
• Files with sensitive data, passwords, or usernames; and
• Information about web servers.

5. SpiderFoot

SpiderFoot automates collecting information about IP addresses, domain names, e-mail addresses, usernames, names, subnets, etc. and comes with an opensource version. This tool allows you to examine any suspicious IPs, phishing scam e-mail addresses, and HTTP headers (which can be parsed to reveal OS and software version numbers, etc.). It’s also useful to organizations for monitoring any information that’s been made public inadvertently.

6. Foca

A network infrastructure mapping tool, Foca, can extract and analyze metadata from different types of files (pdf, doc, etc.) fed at a time or given together. It also can enumerate users, e-mail addresses, software being used, and other useful information.

Need Additional OSINT Tools and Resources? Look No Further…

We’ve put together a list of other resources that may assist you in carrying out any OSINT research includes (but is not limited to) the following:

• Plugins — Passive Recon is a Firefox plugin that searches through several public databases and look-up services. It passively gathers information about a domain so long as you’re on the page or one linking to it.
• Search engines — Apart from Shodan, attackers and pentesters frequently rely on search engines like Censys, ZoomEye, Greynoise, BinaryEdge, among others.
• E-mail harvester — Some tools like the hunter, theHarvester, Prowl, and a few others can help you detect email information for a target organization’s employees.
• DNS Enumeration — Tools like DNS dumpster, Sublister, etc. are useful for enumerating valid subdomains.

Additional resources include haveibeenpwned, Recon-ng, CheckUserNames, Creepy, Nmap, etc.

Final Thoughts on Open Source Intelligence Gathering

Frankly, there’s a lot to know when it comes to answering the question, “what is OSINT?” As such, I hope this article provides clarity about open source information, open source intelligence, the OSINT framework, and showcases the types of OSINT tools that are available to you.

Although you can use OSINT techniques to cyberstalk or conduct other nefarious deeds, you can also use them for good purposes like fuddling information and misleading attacks to protect privacy. Any data that’s made publicly available will be accessible in bits and pieces to anyone with or without the knowledge of OSINT. With it, an individual or an organization has the tools necessary to assess what’s out there and, at the very least, obfuscate the narrative.

• #open source data
• #OSINT