Datasets not listed in API query

[likeaword]

title:When making an API call some datasets don’t appear: unexpectedly
slug: API-mystery
username: likeaword

(note: username should already exist on DataHub.io)

Hi we’re building a basket of sustainability indicators. We’ve using the datahub to hold the data and we’ve been working with a local developer to build a visualisation tool. This tool pulls data from the datahub via the API but datasets are not appearing via the API when we expect them to (and the appear in the UI.

The specific API call is https://datahub.io/api/3/action/package_search?fq=organization:hdata%20AND%20Indicator:*%20AND%20Better:*

It pulls some data but not all the data.

Is there something we’re missing?

Anyone got any ideas?

[mattfullerton]

Hi!

Could you provide an example of a dataset that isn’t showing up?

It might be easier to use a tag to group things together into what you want and then grab them like that with the API?

http://docs.ckan.org/en/latest/api/#ckan.logic.action.get.tag_show
(using include_datasets: true)

By the way, we are building up a project monitoring the UN SDGs here and hoping in the future to make a version for local applications (the system is very flexible about what indicators are used): https://2030-watch.de/en/ - perhaps there’d be some room to collaborate?

[likeaword]

Hi @mattfullerton

Thanks for the reply.

An example dataset that isn’t being pulled through is https://datahub.io/dataset/number-of-people-in-herefordshire-accepted-as-homeless We are using a custom field of “Indicator” to group indicators. Is there any advantage to using a tag instead?

We’d love to collaborate. Separately I’m also involved with the ODI node in Cardiff, Wales. The Welsh government has new legislation requiring the evidencing of sustainable development by all public bodies in decision making (that’s a really poor summary) Children and families | Topic | GOV.WALES

This is our draft tool. https://commanderturner.bitbucket.io/ If we can solve this API issue, we’re planning to put it out for live test locally. Happy to share our learning…

[likeaword]

Had a quick play.

On the face of it using a tag seems to give the right results (24 datasets are currently tagged “herefordshire” and this https://datahub.io/api/3/action/package_search?fq=organization:hdata%20AND%20tag:herefordshire seems to return 24 datasets.