December 23, 2020 0

What Is a URL Blacklist?

in Cyber Security

(3 votes, average: 5.00 out of 5)

Loading...

A URL blacklist is a big issue for site owners & businesses. We’ll explore what a URL blacklist is (including Google’s blacklist), what its purpose is, and how to get your site off one…

What is a URL blacklist? Before we can answer that, let’s consider that the internet was once like the wild west — black hats and cybercriminals ran amok and wreaked havoc. Fortunately, however, governments and other authoritative bodies have (slowly but surely) implemented standards and boundaries to help police and secure our beloved internet. One such example is the PCI Security Standards Council and Payment Card Industry Data Security Standards (PCI-DSS). After years of a worsening credit card fraud problem, five major credit companies created the council to enforce the Payment Card Industry Data Security Standards (PCI DSS) and ensure users can safely use their payment cards over the internet.

This kind of policing can be found in many other spaces of the internet. A URL blacklist is a great example of that. A URL blacklist, basically, is a list of websites that are deemed insecure or dangerous by a search engine, antivirus program provider or another authoritative body. When a site joins a URL blacklist, it comes with some steep consequences for the website owner. Just like PCI DSS, this is a way for search engines, antivirus service providers, etc. to maintain some level of accountability and ensure that websites are held to a security standard.

If you’re a website owner, are interested in creating a website, or if you’ve found your site on a URL blacklist (yikes), then you’ll want to know more about this topic. In this article, we’ll answer your questions like “what is a URL blacklist?” what it means for your website, how you end up on one in the first place, and what you can do to fix the issue.

What Is a URL Blacklist? Blacklist vs Blocklist

In general, a blacklist could include everything from specific websites and IP addresses to entire domains. So, a URL blacklist is a list of specific URLs/websites that are categorized as unsafe. It also should be noted that Google is moving away from language like “blacklists” and “whitelists.” In some cases, these terms are being replaced by words like “blocklists” and “allowlists.”

But regardless of what they call it, this may leave you wondering how or why you may suddenly find your site on such a list. When placed on a URL blacklist, your URL is essentially removed from the indexes of search engines, antivirus services, etc. One of the most commonly known URL blacklists is the “Google blacklist,” which is better known as the Google Safe Browsing list. If your website lands on this list, you are in for a world of trouble.

What Does Being Blacklisted Mean for The Website?

Your website will be blocked on not just one but two or more web browsers. The obvious one is Google Chrome, but your website will also get blocked from Mozilla Firefox (due to Google’s partnership with the browser) and Apple’s Safari browser as well. These are among the most used browsers in the U.S.

But what does it look like to website visitors if your site winds up on the naughty list? Users will see these warnings if your site is on the Google Safe Browsing list:

In addition to users being blocked from your website via the aforementioned web browsers, you’ll also not be able to run Google Ads, which will hurt your ability to drive traffic. To make matters worse, even if a user is running a different web browser and uses a Google search, there will be a warning message next to your website name that reads something along the lines of “this site may harm your computer.”

And finally, the cherry on top: your website will be flagged if the link is sent to a Gmail address or if a user tries to click through to your website from a Gmail address. A nightmare, indeed.

Google Isn’t the Only Company That Has a URL Blacklist…

Simply put, Google is a far-reaching, massive enterprise. So, it makes sense that many web security programs adopt the Google Safe Browsing list as their own URL blacklist. But Google isn’t the only one to venture into the area of URL blacklists and website ban lists. In addition to Google Safe Browsing, there are other authorities that have their own blacklists that you should be aware of.

• There is the Norton Safe Web URL blacklist, which basically means that users will be blocked from your website and receive a warning message if the use Norton Antivirus or the Norton Safe Web application.
• Similarly, McAfee uses their own blacklist for the McAfee WebAdvisor program. Being on this blacklist means that users who deploy McAfee applications will be blocked from your website.
• Another search engine that carries a URL blacklist is Bing. If you land on their URL blacklist, users will receive a similar warning (as Google) when trying to reach your site through their search engine.

How Does a URL Get Blacklisted?

We’re so glad you asked! Search engines and antivirus program providers have made it their duty to ensure users that browse while using their applications are safe. It’s obvious why a company whose purpose is to provide web security would prioritize that. As for search engines, they want to deliver the best user experience possible, and leading users to unsecure websites doesn’t exactly scream “great experience.” The criteria for being placed on a URL blacklist were created under this line of thinking.

Essentially, a website will land on the URL blacklist if the organization blacklisting the site believes it:

1. Was created for a malicious purpose,
2. Has been hacked (and is then considered unsecure or dangerous for users to engage with), or
3. Has deceptive intent.

Okay, we’re going to assume you’re not a hacker running a phishing scheme through your website and that you’re a genuine site owner who’s looking to get off a blacklist. This means you fall into the second or third category.

Although, it should be noted that some blacklists, namely those of antivirus providers, will blacklist a URL if there are enough users who report the website as spam. So, if you do wind up on a URL blacklist, it’s worth taking a hard look at your website to ensure there isn’t anything that would come off as immediately suspicious, such as:

• Broken links or an abundance of redirects.
• Too many ads.
• Missing or nonsensical content — i.e., literal random, blank spaces on your page, not enough provided info on your business (privacy policy, about page, etc.), or a bunch of random words shoved together.
• Content that looks out of place (such as the actual code shown on your website instead of what it is intended to display).

Also, be sure to take a look at any click-tracking services you may use in emails. Even legitimate businesses can find themselves on the Google URL blacklist through using services that are affiliated with redirects to phishing sites.

Now, back to the two main reasons you can find yourself on a URL blacklist…

Your Website Has Been Hacked

One of the most common ways a website gets put on a URL blacklist is because the site has been hacked. And many times, the website owner doesn’t even know it. This could be due to many reasons, including:

• Using weak, easily guessable passwords,
• Not updating or patching your website or software (which may leave vulnerabilities in your website that hackers can exploit), or
• Someone having access to your website through other means (such as through the use of compromised credentials, which they could have gotten by sending you or one of your employees a phishing link).

What Is Happening When My Website is Hacked?

There are a few things that a hacker could be doing with your website…

1. Your website displays content that’s visible to others but not you. The hacker may show content that is typically viewed as distasteful or markets something that has nothing to do with your website. But what makes this even worse is that they hide it from you but allow other users to see the altered version. This allows them to try to trick users into engaging with their fake content while you’re none the wiser about what’s happening.
2. Your website may be used to spread malicious content or software. The hacker is using your website as a way to spread malware. They may do this through your links, downloadable attachments, dynamic content, or even when a user simply visits your site.
3. The hacker may use your website to attack other websites. For example, they could use your website as a part of a massive group of websites/servers called a botnet. Then the hacker could use the botnet to launch DDoS attacks on other websites.

Deceptive Tactics

Another reason you could end up on a URL blacklist is due to deploying some sort of deceptive tactic. Whether it was done unintentionally, or you knew it was deceptive doesn’t matter. What does matter is that you’re going to lose traffic — potentially the majority of your traffic depending on what URL blacklist you end up on — unless you fix the problem.

Some of the deceptive tactics that could get you flagged and blacklisted include:

• Not clearly communicating you are using a third-party service. If your website utilizes a third-party service, or if a third-party is operating your website for you and this relationship is not disclosed to website users, it’s a big issue. When using third parties to run your website or to provide a major service (like payment management), it’s best to be transparent about the partnership and inform users.
• Deceptively luring or outright tricking users into doing something they are unaware of. If a user doesn’t know they are sharing certain information with your website or is downloading something that is falsely claimed as necessary, this could be considered deceptive and get you blacklisted. Not properly disclosing your cookie policy could fall under this category.
• Altering a user’s personal computing experience. Another no-no is that if the software that your website provides unknowingly alters a user’s browsing or computing experience. This could mean that your software changes a personal browser setting without a user’s knowledge or approval.

How to Get Your Website Off a URL Blacklist

If you think your website is being affected by a URL blacklist, here is what you do…

Find Out If You’re Blacklisted & Where

There are many tools you can use to see if your website is on a URL blacklist. Google Safe Browsing allows you to easily check on your status with them. There are also free tools like VirusTotal that scan for many of the blacklists mentioned in this article. Get a glimpse of their reporting style below:

Fix the Problem to Ensure It Doesn’t Happen Again

Once you discover where you’re blacklisted and why, it’s time to roll up your sleeves and fix the issue. If the issue is malware, there are plenty of ways to remove malware. One of the most efficient and effective ways is to utilize an automated malware scanner & removal tool. A tool like CodeGuard, for example, includes an automated malware scanner and remediation tool (plus it’s an automated website backup tool).

As far as ensuring the hack doesn’t happen again, there are also automation tools that can help with this. I encourage you to read some of the related articles below, which cover how to purge your website of malware and prevent future hacks from happening:

• Have I Been Hacked? How to Know & What to Do Afterwards
• What to Do If Your Website Is Hacked
• How to Secure a Website: 21 Website Security Tips for Businesses

Submit Your Site for Blacklist Removal

Once you resolve the problem, you must now contact the service provider who blacklisted your website and request its removal from the list. For example, if you end up on the Google Safe Browsing list, this is what you do:

1. Create a Google Search Console account.
2. After signing up, you’ll have to verify your website (do it).
3. Once verified, go to the Security Issues section in your Google Search Console account (you’ll find it under the Security & Manual tab).
4. There, you’ll need to click on Request a Review.
5. Finally, fill out the form, explaining how you solved the problem and, if applicable, how you implemented a defense that will prevent the issue from happening again. 
6. If you run into the same issues again, Google may limit your ability to request a review to once per 30 days.

The process isn’t too much different for any other service provider that can blacklist your site. Take Norton Safe Web for example. You basically follow the same steps — create an account, verify your website, and submit a “site dispute.” There are also automated tools that monitor blacklists and attempt to remove your site from them automatically.

What Is a URL Blacklist — A Final Word

We hope that we answered the question “what is a URL blacklist?” and why they exist. Our goal was also to provide you with some helpful tips on how to avoid the issue in the first place, and how to get off a blacklist if you find your site on one. As you can see, URL blacklists can cause a lot of problems for businesses but landing on one is a potentially reversible process — so long as you know the steps to follow.

So, with all of this in mind, remember to keep your website secure, honest, and to provide as much information about your website as possible to your visitors. Good luck!

• #URL blacklist