Status of Scheming extension

CKAN General
1

Can anyone tell me what the status of the scheming extension is. The idea to define the properties of a dataset/package with its data types and validation logic seems perfectly logical to me.

It seems to be a quite central extension. A lot of other extensions seem to rely on the presence of the scheming extension. However, the CKAN demo site https://demo.ckan.org does not use scheming.

So far, we have added additional fields manually in the code and templates (in our own extension). Switching to the scheming expansion will surely clean up our code but is likely to be quite a big undertaking. That’s why I’d like to make sure I’m going with the right option here.

2

Hi @jze , the scheming extension is indeed a central extension - allowing you to configure your metadata schema via a yaml file.

We use it in ALL our CKAN sites.

As to demo.ckan.org not truly being a fully-loaded demo site - one of the key goals of this CKAN Ecosystem Catalog is to eat our own dogfood and serve as a demo of CKAN itself, and scheming is how we are able to catalog Sites, Extensions and Tools beyond the traditional dataset.

Check out how we’re doing it here -

cc @a5dur @wardi

2 Likes
3

@jze The core team has discussed merging ckanext-scheming functionality into core, but there are benefits to keeping it separate too: more frequent updates and new features in the hands of users sooner.

ckanext-scheming is one of the best-tested extensions you can count on to stay up to date. FYI github.com/ckan/ckan/issues/9364 is the ticket tracking core compatibility for ckan 2.12, being released next week.

2 Likes
4

Adding to @jqnatividad’s post, here’s how we’re using scheming for the Ecosystem Catalog.

on the catalog we have separate custom types for extensions and sites, each with its own schema:

@jze on the “big undertaking” worry:

You don’t have to do it all at once. Scheming works per dataset type, so you can stand up one type on a schema, leave the rest on your existing code, and migrate as you go.

If your custom fields are already stored as extras, matching the field names in the YAML is often most of the work. Worth checking how yours are stored before you plan the migration, since that changes the effort a lot.

The escape hatches are good. Between presets, form_snippet / display_snippet, and custom validators registered from your own extension, we haven’t hit a case where we had to drop back to hand-rolled templates. Most of the logic that used to live in Python moves into the YAML, and what’s left is a small validators file.

One thing that isn’t obvious from the docs: the schema is also your documentation. (though you got to look up)

1 Like