The open data movement is a juggernaut. In the years since the Freedom of Information Act, citizens have come to expect access to public information of all kinds in the easiest, fastest possible way.
At first, federal government agencies struggled to make their information available to the public. Data was typically old, inconsistent, and formatted in unwieldy ways. Decades of pressure from government transparency and accountability advocates, however, have raised expectations for timeliness and usefulness of government data. These higher expectations have seeped down to the state, county, and municipal levels with the result that the quantity and quality of open data have never been higher.
Not everything is rosy, though. The march of progress toward ubiquitous, robust, and timely open data is now confronting perhaps its biggest obstacle: Accurate data makes some people look bad. This wouldn’t be a problem if not for the fact that the people most likely to come off looking bad are the very people required to furnish the data in the first place.
“Looking bad” when data on your work is analyzed is not just unwelcome news to these government employees. It can affect their livelihoods if they’re fired or not promoted. It can even expose individuals and agencies to lawsuits. Imagine the simple case of deciding where to place a fire station. If a municipal employee’s data analysis leads to putting the station someplace where they can’t reach certain neighborhoods quickly enough, then people can actually die. Most jobs do not have this life and death aspect to them, but many government jobs do.
Ted Lehr, chief information architect for the City of Austin, has been the pied piper for open data in our fair city and he recently observed that risk aversion in government work is a fact of life. This means that the move to “born open” government data will likely face an uphill battle as we get closer to 100% openness. The last holdouts of government data repositories (most likely public safety and public health agencies) may only become open when the fallout from the inevitable analyses of those data can be mitigated in some way.
Also, there’s no universally accepted model for the recovery of the government’s expenses in making their data accessible. An agency in one county may offer a given set of their data for free as an API while another county charges thousands of dollars for a DVD of the same data.
There is good news for publishers and other value added resellers of government data in all this: without a universal model for access to data, there will still be a place for aggregation, normalization, curation, and independent analysis.