Scraper Directories

Also known as scraper spam or simply as scrapers, scraper directories are considered by many webmasters to be the internet's bottom-feeders. If you came across a long, odd-looking page full of links that all tie in with a certain phrase that looks more like something you would type into a search box than something you would actually say, the chances are you are looking at a scraper directory.

Most scraper sites are characterised by their thousands, and in some cases millions, of pages. Each page consists of links to a large number of pages from other people's websites. All of these links are accompanied by a snippet of text that has been copied, or "scraped", from the other website's page.

Another giveaway is the navigation bar, which will often be extremely long and consist of words and phrases that are closely related to that page's main keywords. Every page in the scraper directory is used to target a particular keyword or keywords. So all of the snippets from the other sites will mention these keywords in some way.

Scraper directory pages are all created automatically, without human intervention. The result is often gobbledegook.

For instance, a scraper page on "apple growing" might include snatches of text about the growth figures for Apple corporation, a description of how the adam's apple grows in size during puberty, and a page about horticulture. No good directory would lump these topics together.

Something else that's common to most scraper directories is that they're unlikely to contain any original content. Instead, the text is all copied without consent from other sources. This often puts them right on the borderline between legal fair use and copyright infringement.

Why bother?

You may be wondering why anyone would bother to produce thousands of pages of legally dubious and tiresomely repetitive nonsense. It's a fair question.

The root of this particular evil is money. Scraper directories exist to exploit loopholes in the search engines. Because search engines aren't human, they have a hard time distinguishing between sense and nonsense. They rely instead on factors such as the number of links to a page, the number of times a word is used, and other factors that can be compared mathematically, in order to determine their results.

Scraper directories aim to make money by putting themselves between search engine users and their destination. It is frustrating to click on a search result only to be taken to another page of yet more search results. This is what scraper directories are all about. The directory owner wants to make the visitor leave immediately by clicking on an advert. The payout may only be a few pence each time this happens, but that hardly matters to someone with a 40000 page website.

The owners of these directories don't care about producing relevant results or interesting articles. They don't want people to stick around to be entertained or informed, and they provide no useful service. That would cost money. In short, scraper directories are a waste of pixels, run by people who want to make money by wasting your time.

What you can do about this

If you land on a scraper directory when you are surfing, do not click on ANY of its links. It is common practice for some of the links in these directories to be advertising links disguised as regular ones. They may be the same colour as ordinary links, or mixed in with the rest of the text so that they don't stand out. If you click on the adverts you are ensuring that the owner of the scraper directory gets paid.

Moreover, clicking on the normal links on these sites will often cause the address of the scraper directory to appear on the statistics pages of the website you end up visiting. Sometimes these statistics pages are made available to the public. So by merely using the scraper directory, you could be helping to promote it.

Essentially scrapers are a problem for search engines, since these useless websites would not exist without them. But by refusing to use these websites we can ensure that the owners do not get paid for annoying people.