News Explorer collects and displays news articles where wildland
threats are discussed on the web. The application is hosted in
the Amazon cloud and utilizes the Google AJAX Search API to create
a database of news articles for specific predefined threats such
as bark beetle, climate change, etc. There is also an option to
enter search terms and retrieve a list of articles retrieved from
a Google news search.
The Threat News Explorer
(TNE) application was built for the USDA Forest Service Western
Wildland Environmental Threat Assessment Center (WWETAC). TNE
facilitates assessment of wildland threats by collecting and displaying
news articles on the web that discuss these threats. The user
can view the latest, all, or selected articles for specific predefined
threats such as bark beetle, climate change, etc. There is also
an option to enter your own search terms and retrieve a list of
articles retrieved from a Google news search. Articles retrieved
from a user specified search are not saved.
The application is
available at http://www.wwetac.us/TNE/TNE.aspx.
Using the TNE application is simple. Users have a choice of either
selecting predefined wildland threats, or entering their own search
terms at the top of the page.
The application display
area is divided into two sections. On the right is a map displaying
the locations of places discussed in the articles. The left side
of the display contains a tabbed interface with the following
tab - This is where the articles are displayed to the
user starting with the most recent articles. Each returned article
provides a direct link to the article on the web, as well as
the name of the publisher, the published date, and a short summary.
Below the article summary are the names of places discussed
in the article. Users can click on a place name to identify
the location on the map. The map will zoom to the location and
display a popup window for the place.
tab - The number of articles per day are displayed
in an interactive line chart. This chart is created with the
Visualization API. The chart applies to articles with places
within the current map extent and automatically updates as the
user pans and zooms in the map.
tab - The select tab has three main options:
- Displays all articles collected for the selected threat
since the TNE application was deployed.
- Displays the most recent articles for the selected threat.
- Provides a way for the user to select articles by specifying
a published date range and/or search term. The user can
choose to search for the term in the article title, publisher,
summary, or the entire article.
The application is
hosted in the Amazon cloud and utilizes the Google
News Search API to create a database of news articles for
specific predefined threats. Each threat is searched on a daily
basis and the results stored in a database on the TNE server.
The server makes a copy of each news web page for internal use.
During the process
of collectiong articles, the TNE application also searches for
known places discussed in the articles. This process uses Yahoo's
PlaceMaker web service for identifying places in the web page
text. The process of identifying places relevant to the article
can be difficult. It is possible for the PlaceMaker web service
to simply make a mistake in identifying a place. We attempt to
remove incorrect places or places not relevant to the article
through an internal ranking and filtering system, and also through
manual edits to the database. The places are displayed on the
map as points. The user can click on a point to view a popup window
for the place. This window will list each article that discusses
this place, along with clips of the article text that contains
the place name.
This project was inspired
by the paper written by Galez, et al (http://www.scribd.com/doc/13756691/Galaz-et-al-2009).
This project provides the basis for working towards the concept
presented in the paper by starting with a web crawler and data
archiving system for news articles on wildland threats.