Testing New Search Tools on Government & Campaign Information

Back in the day, when Yahoo! was the only search game in town, many wondered why Ask Jeeves (now Ask.com), and eventually Google would attempt to break into that market. The answer continues to be the same - although they're good, there's still a lot to be done with Search. Contextual search is still being explored, and in terms of government and campaign information, most documents are not publicly or easily available to the search engines. With the goal of open government in mind, I decided to take a look at five relatively new search companies that recently launched sites, hoping that perhaps some of them could help make search of government and campaign data a little better, honing in on the FEC, OMB and more.

Here's a brief look at what they can do...

1) Cuil - (pronounced "cool") went live at the end of July. It has an interesting user interface (UI) that shows a portal-like visual of information found for each topic. The customizable view allows for varying screen sizes and it's easy to read.

2) Kosmix - aims to "organize the web". Born in 2006, it's still in Alpha, taking on verticals with its unique look. Well presented data and intuitive subdirectories, this site makes search seem less daunting.

3) SearchMe - "visual search" is in Beta. Around since 2005, they launched the first version in 2007 and went into private Beta last March. Now public, it requires a Flash package to be viewable - users can see images of pages at a glance. There's also an an iPhone application that searches visually. It can be slow, but pages are immediately expandable from within the search.

4) Deepdyve - (formerly Infovell) just released in November, this is a "deep search" contextual and document based search focused on the "Deep Web". They make agreements with and troll parts of the web not searchable by most others. The site currently requires signing up to use the service. (Disclaimer: my husband previously worked in Program Management there.)

5) Viewzi - recently advanced to public Beta, it provides a variety of data views, including the Power Grid View and the TechCrunch View. Loading data from Yahoo and Google, it essentially aims to make the data look better, with a variety of angles in which to read and process the information found.

For an initial test, I searched for Federal Election Commission because many journalists look at FEC filings to obtain information about candidates and campaign spending. The FEC site and opensecrets.org provide both easy interfaces for finding information, so this is a good comparison point. Cuil provided a variety of Advisory Opinions and a limited list of presidential candidates related to FEC. Kosmix, under "Related in the Kosmos", 8 categories were listed including Campaign Finance, Election, and the Bipartisan Campaign Reform Act of 2002. SearchMe brings up two visual stacks in a similar way to the Mac Leopard document viewing system - Campaigns & Elections and US Government. It brought up some interesting documents and moved quickly sifting them, but it took a little adjusting to the interface.

Deepdyve came up with a few interesting results and a variety of categories. When I refined the search, removing "life sciences", "physical sciences" and "clean tech" from the criteria, a Huffington Post article disappeared that was relevant. With Viewzi, the Google Timeline view of FEC data honed in on a Politico article about Obama escaping an FEC audit. The Simple Text View soon found a Report of Receipts and Disbursements for Hillary Clinton for President.

Next, I looked for anything related to the OMB (Office of Management and Budget). SearchMe brought up 4 cute little icons for US Government, Labor, Politicians and Blogs, but the pages presented under each option didn't vary much. Kosmix found "Our Miss Brooks" and Tomb Raider before the White House. Cuil came up with some good links. Viewzi went straight to relevant docs. Deepdyve located a few Annual Review papers.

Then I took it to another level, searching for "open government presidential directive technology infrastructure" and where Google came up with the Homeland Security Presidential Directive 7, Cuil pulled up a blog post from Tech Insider at GovernmentExecutive.com. Kosmix came up with the same directive 7 as Google, but provided several related categories worth searching. SearchMe located PediaView's definition of a presidential directive, and Viewzi found a report at businessofgovernment.org about "Strengthening Homeland Security." Deepdyve listed a Global Business Review article entitled "Transformation of an Emerging Economy to a Knowledge-Based Economy: Korean Case" and later a Public Works Management & Policy article about "Rail Security After 9/11."

Taking advantage of Deepdyve's document based search capabilities, I copied the Abstract of Presidential Directive 7 and pasted it into Deepdyve to see what would happen then. At that point, it brought up the rail security article first, followed by the Wikipedia entry on Critical Infrastructure Protection. Pasting the entire abstract into SearchMe brought up a Memorial Day open thread and a Canadian blog. Kosmix seemed confused, but it still found Directive 7. Cuil found a TSA document on Rail Transportation and a couple of GAO pages. Viewzi zoned in on last year's National Strategy for Homeland Security. Google went straight to Directive 7.

As to whether these new search engines have made any progress in contextual government search and whether any of them are helping us open government or improve media access, it's hopeful. If we could import a greater depth of government documents into a system like Deepdyve or SearchMe, particularly with sophisticated subcategorization, we might have some powerful tools. With all of the talk about the new administration reaching out and providing more open access, it should follow that more information will become publicly searchable. For anyone wanting to follow up on this, these sites are great places to start looking for ideas and suggestions to make to the new administration as they build out their technology and new policies.

Comments

Great piece, Sarah!

Loved the post, Sarah--thanks for your thoughts on the space.
I'm part of the Kosmix team, so it was intersting for me to see the results of your comparision, particularly from the open government perspective.

Worth noting that Kosmix has evolved from our roots as a vertical search engine to become a horizontal guide to the Web. We don't see ourselves as a search engine, per se, because search engines are designed to help you find specific facts and navigate to sites you already know. Kosmix is all about getting a broader perspective and discovering what else is out there.

Or, to put it another way:

Search + Find = Google/Cuil/Searchme
Explore + Browse = Kosmix

Youtube

The latest contributors to

The latest contributors to the national layoff numbers has been Adobe who has cut 600 jobs, AT&T which is cutting 12,000 jobs, Viacom who laid off 850 and NBCU which is letting 500 people go. These are scary measures, for scary times and unfortunately a necessary evil for companies to stay afloat in today's upside-down economy. Consumers, who are paying the most, have been feeling the effects of our economy's fallout since the Federal Reserve stepped in and bailed out many of the nation's banks via American consumer Taxpayer dollars. This was the first big hit the American Taxpayers had to take. Now, that the banks have been extended a hand of Federal mercy, they now begin to clamp down on personal and business financing which has only made it harder for both everyday consumers and business owners who, if not allowed to spend, or operate cannot stimulate the economy. Consumers with good credit have had a difficult time obtaining approval for home loans, auto loans and a variety of other payday loans. Banks have tightened their belts to cut back on the mortgage defaults, which were and still continues to be one of their greatest problems to overcome. Unfortunately for the consumer, the banking industries consistent endeavors to raise banking fees tax the American economy even more. Overdraft fees, minimum balance fees, ATM fees and other miscellaneous fees are among those that have been increased. It has been perceived that the banks are raising their rates to compensate for their losses in revenue due to mortgage defaults. If there is anything good that has come out of the economy, it would be the fact that America's consumer base, which has been so prone to overspending and borrowing more than they can afford, have now woken up to the reality and constraints of personal debt. We rank number one in the world when it comes to consumerism and debt, so if it doesn't break this country, it will only make us better. Until stability comes, you may find yourself needing a little extra cash. Click to read more on payday loans.