Open Government Data - where to next?

Open Knowledge Network Australia
1

In a recent discussion with David Read within the CKAN dev email list I put down some thoughts in relation to something I say quite often. For those who don’t know what CKAN is, it is the open source project that is used to support open data portals such as data.gov.au and many others in Australia and around the world. Anyhoo, I made comments about how I recognise machines as the primary users for CKAN, custodians the secondary and end users are tertiary.

David made the fair comment that in practice the number of actual machine users are small. He also asked where I saw open data catalogues going in the future. The response I provided follows.

Once setup you’d like to load or register datasets in a catalogue. The data will typically come from an external source of truth. Whether a DB or a spreadsheet, or a harvest location. Moving/registering and maintaining the currency of those datasets is best done with automation. This is where I think the back end machine to machine use cases are extremely important.

From a front end we already have resource views with embed scripts. This is a great example of supporting another machine to machine integration point. Although it sounds technically trivial, having a view on a data resource embedded on any other website supports real time syndication of those views.

From an open government perspective where transparency around government business is important, I see the future of data catalogues as being the window into operations. A place from which anyone can draw government business intelligence. Government can build data triggers into every day operations and publish these events into datasets in a similar way as google analytics does for website events.

For example, Government can easily build simple web form integrations with CKAN via any CMS to run public consultations where community feedback is published in real time. This can make it more clear when lobbyist or activist groups flood responses.

I’m a big supporter of machines as primary users, perhaps because I see open data as a way to iteratively transform government operations by showing where improvements can be made.

I’m working on how to grow these ideas around open referral, open 311 and CKAN to provide open platforms for transparent and accessible government operations. I think things are ready to come together quickly and the next few years will see open data catalogues receive a lot of attention.

So, with this re-post I thought it would be good to see what others think about the role of open Government data within the Australian context and where we’d like to see more progress. We know there are key datasets we’d like released, but what about a more holistic view on the principles for open Government data inasmuch as it relates to the principles of open Government - transparency, participation and collaboration?

3 Likes
2

Hello Steven,

In my opinion, it’s important to view the economy as a whole when analyzing the budget of the government. It’s important to analyze the broad economy because different fiscal strategies work in different environments.

The goal of my website econfactbook.org is to make the world economy easier to monitor. It has statistics from many online sources and has citations for all of the data. Here is my page dedicated to the economy of Australia. I hope this is the type of website you are looking for. I’m trying to make data more transparent also.

3

I disagree with the idea of prioritising user groups:

  1. machines
  2. data custodians
  3. data re-users

CKAN is a platform designed to connect all these groups. Like other platforms that generate value from connecting consumers and suppliers (think Uber, AirBnB), everyone should receive value from using the platform.

(People instructing) Machines would like:

  • Well described and stable APIs
  • Guaranteed service levels
  • Machine readable everything (and @Starl3n, I like your embedded views/tools ideas)

Data Custodians would like:

  • Automatic data quality checks
  • Automatically derived metadata
  • Simple (re-)publishing workflows so they can keep the promises made in the metadata

Data Re-users would like simple ways to:

  • visualise/analyse the whole dataset (e.g. Open in Tableau, CartoDB, etc.)
  • analyse multiple datasets and look for relationships in the data

As we move to Government as a Platform, do open data portals (like CKAN) get side-lined?

1 Like
4

I might agree with you there @Stephen re using the priority list. Rather, the foundation upon which a great end user experience is gained comes from a flexible and powerful API. With regard to data custodians I think also that the logical structure of CKAN’s publishing paradigm should remain linked closely with the structure of Government operations. Jurisdictions > Departments > Programmes > Outcomes.

All points are highly relevant so thanks for providing your thoughts on all this.

1 Like