• Italy | Is transparency compatible with “robots.txt”?
  PDF friends David Osimo and Alberto Cottica point us out a story from Italy about a “transparency project” launched by the Italian government.
  The initiative, launched some time ago, aimed at publishing relevant information about civil servants, such as paycheck and days of absence. But, as this article points out, most part of this data (including those about the ministry itself) has been published in a directory which is not possible to reach by search engines – using the robots.txt file with “disallow:/operazionetrasparenza/”.
  Here’s David’s take on the story: “The implication is that searching with google the name of a person, you will not find these data. You will have to know that the person is employed by a public administration, and visit the website and check the name. This is obviously limiting the real transparency of the public data.
  I assume the excuse is related to privacy: there are different privacy implications if a personal information is searchable or not. This is an important matter, which I would like to understand better. Yet in this case it appears as an excuse. Real transparency needs machine-readable data, and using robots.txt is a clear contradiction of the principle of transparency."
  Plus, David has another point to make: why is transparency applied first of all to (against) public sector workers and their behaviour instead on how the P.A. spend public money?

Back in the day, when Yahoo! was the only search game in town, many wondered why Ask Jeeves (now Ask.com), and eventually Google would attempt to break into that market. The answer continues to be the same - although they're good, there's still a lot to be done with Search. Contextual search is still being explored, and in terms of government and campaign information, most documents are not publicly or easily available to the search engines. With the goal of open government in mind, I decided to take a look at five relatively new search companies that recently launched sites, hoping that perhaps some of them could help make search of government and campaign data a little better, honing in on the FEC, OMB and more.