The price of digitization

(this page is part of my 2011 report on “Open Data: Emerging trends, issues and best practices”. Please follow that link to reach the Introduction and Table of Content, but don’t forget to also check the notes for readers! of the initial report of the same project, “Open Data, Open Society”)

In practice, public data can be opened at affordable costs, in a useful and easily usable way, only if it is in digital format. As a consequence of this fact, demand for Open Data exposes a problem that already existed and must be fixed anyway, regardless (again) of openness. Any substantial increase of efficiency and reduction of the costs of Public Administrations can only happen when data and procedures are digitized. The problem is that such digitization (which, obviously, must happen anyway sooner or later) can be very expensive and we are only now starting to really realize how much. Actual, material costs are not the worst problem here. Activities like semi-automatic scanning of paper documents or typing again their content inside some database, are relatively low, one-time expenses that are also very easy to calculate and budget in advance with great precision.

The real costs are those at the social, cultural, historical and workflow reorganization level. What is really difficult, that is expensive in ways that are hard to predict, is to fit inside digital, more or less automatic procedures and file templates, formats, habits and customs developed, maybe over several centuries, in the analog, pre-computer world. Developing countries are good case studies from this point of view, because they are often leapfrogging from oral tradition straight to computers in all fields, not just e-government.

Land ownership in India, discussed by Gurnstein in 2010, is a perfect example of the problems carried by digitization that requests for Open Data only expose, without creating them. Digitization can certainly increase efficiency, transparency and economic activities, but fully achieves these goals only by:

  • standardizing as much as possible all concepts, formats and procedures.

  • replacing completely, at least in standard day to day procedures, whatever other records and ways of working existed before

Gurnstein wrote:

_"The problem of open access in the case of land records in India is... the manner in which the data tends to get encoded. Typically, digitization of land records would mean either scanning the record as it is, or inputting all the data on the record as it is, without changing any fields. But ways of maintaining land records are highly diverse... Private ownership is not the only means of holding a land parcel. When it comes to land ownershipt, for example, it may eliminate the history of land, how were sub-divisions and usufruct rights negotiated and enforced."_

Another risk of digitization and e-government (without openness, that is) is lack of contact between citizens and institutions:

_"Prior to digitization, land records in India were available to people who made requests with village accountants for them. .. after digitization of several services, village accountants no longer personally visit the villages they are in charge of... What has happened with digitization is a reorganization of earlier forms of social and political relations. Accountability has moved from the immediate village level"_

Of course, all these problems existed well before computers and return every time the political or social order changes. The demand for Open Data is only increasing, by orders of magnitude, the numbers of times in which we meet them.