Setup an automated CKAN Repository


#1

Hello.
I was wondering, and I want input and feedback from the CKAN community.
There are so many CKAN data portals out there and alot of them are NOT captured in this repository http://ckan.org/instances/# . This repository serves many purposes including - advertising the progress of CKAN in the open data community, enabling portal administrators to identify each other and connect, a repository for developers etc.

The problem with the present repository is that portals are submitted/included through a manual process. Individuals have to visit the CKAN site and fill a from before their data portal can be included in the repository. As a result, alot of portals slip through.
I am proposing that an automated portal submission path/step be included to the CKAN installation process. During installation of a CKAN portal, details of the portal should be automatically submitted for inclusion to the repository. Users can opt-out of the inclusion if they desire.

So what does everyone think? Opinions and suggestions are very welcome.


Help make the most comprehensive list of open data portals around the world
#2

Neat idea! Automating it would be problematic for those of us running early alpha/beta versions of CKANs that we’d rather keep under wraps though.

In a similar vein - perhaps a form within CKAN itself - that only appears for sysadmin users - which prompts them to submit their portal (and pre-populates as much as possible based on the install).


#3

Wow!! Love this idea!
Thanks for pushing the initial suggestion forward. The use of forms within CKAN that would be available to sysadmin users is excellent. It’s definitely less obtrusive than my initial idea.

Please help spread the word around the CKAN community, so we can push this and get it added to the development cycle. Everyday that this feature is absent, there is a CKAN portal being deployed which the community may never know about


#4

Hi, very supportive of this idea so that we not only have a list of CKAN production instances around the world but can also automatically fold them into DataPortals.org. The only tricky bit would be getting the location (lat,lon) via the form shown to the sys-admin.

Perhaps we could combine the list of the attributes captured in DataPortals.org with the data shown on the CKAN around the world to provide a more complete list.

Are there other attributes that need to be captured?


#5

Yep - this is something I’ve hoped we’d do with CKAN soonish too.

As Keith mentioned it would need to be configured by the platform custodian to be on or off and by default it would be off on install. You’d want it to be able to ‘phone home’ for a few reasons. It could register itself and appear on instances.ckan.org but perhaps more valuable for the platform owner it could return with some understanding about whether it needs some updates to either the core or extensions that it uses.

It would be essential for bringing CKAN up to the enterprise standard where imporatant security patches are made obvious to custodians in a number of ways - email alerts is good but a UI warning will help even more. It keeps the support suppliers honest about whether they are keeping a platform patched as required.

btw - by adding this as a feature for CKAN 2.6, we’d know that previously unknown portals would be likely to become known after they were upgraded. so, we really just need to make CKAN more awesome to encourage platform owners to upgrade.

Cheers,
Steven


#6

Hello @Stephen .

Regarding getting location (lat,lon) from the form, All modern browsers now support geolocation API which can be used to obtained location using a variety of available options e.g. GPS, Cellular Triangulation or IP Address location. If a user’s browser doesn’t support geolocation API or the user refuses to grant permission to the API, we can always provide a fall-back selection list providing a list of countries and their capitals. The user can then select the country of their choice, we use the user’s choice of country to provide a location (lat,lon).

Regarding the attributes, I think the attributes provided can be merged to provide a complete list. So long as many of these attributes can be pre-populated by the form and it doesn’t become cumbersome to the sysadmin-user.

The important question is : How do we get this idea into the development cycle for CKAN? Preferably in time for CKAN 2.6 release. I don’t have a clue how to go about this.


#7

Hello @Starl3n
Completely agree with your views regarding the importance and benefits of including this form into CKAN. I strongly believe that the sooner this form is included, the better it would be for everyone, even the folks currently maintaining instances.ckan.org will benefit from this.

How do we get this included into the CKAN development cycle in time for version 2.6? Do you have any idea? Who are the people that need to be involved/informed on this? Any insight, help or support to pull this through would be immense.
Thanks.

Regards,
Osahon


#8

First step is to just add it to the ideas list on GitHub https://github.com/ckan/ideas-and-roadmap/issues

But take a look and see if something similar is already there - you can then just add comments.

Writing up the use case / story and acceptance criteria helps people to take up the task to start work. Everything is volunteer based right now, with some issues being addressed as possible within paid projects to establish or enhanced existing data portals.

Link Digital would be keen to do this sort of thing, but to be honest there are other things on the roadmap that we should first be contributing to. So, all I can say is that 100% support it but can’t make any strong commitment just now :smile: