Government databases are a treasure trove of random data, but locating much of their content via search engines like Google is often a fruitless endeavor because of a failure to link this data to commonly used search engines, experts told a Senate panel TGovernment databases are a treasure trove of random data, but locating much of their content via search engines like Google is often a fruitless endeavor because of a failure to link this data to commonly used search engines, experts told a Senate panel Tuesday.
Government Web sites like USA.gov, the official U.S Web portal, allow users to search within the site for pertinent information, as do countless other agency Web sites. Their content, however, is not always accessible via search engines, making it difficult for the average Web user to find data hidden in government databases.
An online search for "New York radiation," for example, does not produce basic Department of Homeland Security or FEMA information available on those agencies' Web sites, Ari Schwartz, deputy director of the Center for Democracy and Technology (CDT), told the Senate Committee on Homeland Security and Governmental Affairs.
Similarly, searching Google for "small farm loans" produces commercial offers for loans and government loan stats, but does not include data on federal government programs designed to help fund small farmers, Schwartz said.
CDT and OMB Watch on Tuesday released a joint report that examined the availability of government information on commercial search engines.
Part of the problem is the sheer volume of data produced by the government, said John Lewis Needham, manager of public sector content partnerships at Google. "It is hard to disseminate efficiently," he said.
Government databases typically use search forms that "cannot be navigated by search engine crawlers" so Google has no "means of finding those records," Needham said.
"In talking with Google and other search companies, we try to present the information in a context," said Karen Evans, administrator for e-government and IT at the Office of Management and Budget. "We try to filter out [superfluous data] so we don't create frustration, [but] we might not be doing it in the most efficient way."
One possible fix is deploying the Sitemap Protocol, Needham said. This technical standard was developed by Google in 2005 and allows Web site owners to produce a list – or map – of all Web pages on a site and communicate that information to search engines, he said.
Google has already deployed Sitemap to several agencies like the Office of Scientific & Technical Information, the Government Accountability Office, Library of Congress, the National Agricultural Library, the National Archives and Records Administration and GovBenefits.gov, Needham said.
The Office of Personal Management last week also announced that it would make 60,000 job vacancies in its database available to commercial search engines, said Committee Chairman Joseph Lieberman of Connecticut.
In November, Lieberman introduced a bill, S. 2321, which would extend the E-Government Act of 2002 for an additional five years, and require government agencies to link their databases to commercial search engines within one year.
Lieberman's committee approved the measure on Nov. 14. "I am optimistic we will be able to move this bill through the Senate soon," he said Tuesday.
When asked if the average Web user is even aware of government Web sites like USA.gov, Evans admitted to using commercial search engines for most of her online searches and sites like "USA.gov when I'm looking for specific things."
USA.gov is more than just a Web site, she said, pointing to its integration with the 1-800-FED-INFO phone line. "We tried to put that all together as an integrated channel, [so] it's a more complicated question than does everybody know USA.gov," Evans said.
Evans did concede that "more needs to be done on Regulations.gov," which allows citizens to view and submit comments about federal activity. OMB has been meeting with the American Bar Association on an independent study the group is undertaking about improvements "that would make it easier to use," she said.
Document Management 
