Spiders Have Been Set Lose to Put a Bite on Terrorists

Terrorism research is a branch of intelligence that aims to snoop out terrorists, their cadres, their goals, and their activities. Because terrorists have chosen to use the internet for much of their activities, new efforts have been made to survey the internet for their presence.

These terrorist use the internet to spread propaganda, to recruit new terrorist, to form discussion groups, to obtain finances. In some cases, they have even used the internet to coordinate and facilitate their attacks, to manage logistics, planning, and to share their own intelligence.

The internet is teeming with them; there are thousands of terrorist sites throughout the world, so many in fact that major state governments are investing millions to handle the demand.

These sites do not consist of only one page. If there are thousands of sites, there are millions of pages related to terrorist activity, These pages contain not only text, but digital images, sound, even video. They even present videos to show off their handiwork, such as the beheading of journalists, the execution of their enemies.

Thousands of ugly images dash across the internet lines daily, sparking excitement among their own kind, provoking disgust among the rest of us. With such brazen behavior, it is no surprise that governments have set up internet terrorism research bureaus, branches, units, and have equipped these with the tools necessary to handle such a burgeoning volume.

Because the volume is so great, terrorism research on the web requires more than just some humans surfing the net. How many pages do you think a single person can view in an hour, in eight? Not many when compared with the number of terrorism related pages out there. When the volume is this large, as business has long established, it’s time to automate.

Automated terrorism research is exactly what one of the projects of the University of Arizona’s Artificial Intelligence Lab undertook a few years ago, under the project name, “Dark Web”.

The result is a web spider that locates these nefarious sites, catalogs them, and analyzes them for content. The State Department provides the initial identification of terrorist groups, particular keywords they use, and then an army of spiders are sent out to locate terrorist sites and to trace their links.

Spiders in this context are not an insects. A spider is a program designed to make connections to web sites where they then gather information and return this information to their origin. They are designed to filter web pages for any links to other pages. They will then follow these links, and continue to harvest information on the site and its relations.

The performance of Dark Web is astonishing, collecting as much as 90% of the terrorism content out there. This was not always the case, but by constant improvements, this software spying tool has come into its own. That means we’re winning. Terrorism research has achieved great success, thanks to these scampering spiders that are putting a hard bite on terrorism. Go Spiders!