Hackers.
I’m looking for some pointers on code generation in Python. My use case is around generating APIs from Tabular Data Packages.
There is a Ruby lib that generates a read-only API from data packages, and there are various CSV → JSON API codebases around, including part of the codebase I did for the Open Data Index.
So, straight CSV or Data Package → Read APIs are a great thing, and I’d like to see more in this direction, but, I’m interested in leveraging the power of existing web frameworks, and that is why I’m looking at the next step which will require code generation.
Here is an example flow:
- I have a data package.
- I use the DataPackage lib with the SQL plugin to create and populate a database.
- I generate framework code to manage the data and generate an API (Example:
serializers.py
andmodels.py
and other bootstrapping config in Django REST Framework) - Instant API, but also can hack further around the generated code. The API is also not necessarily read only, so there is the possibility to write changes back to a DP. We also have access to user management, etc of the framework to control API use.
The code generation aspects are step 3 and parts of step 4 (if we went down the route of writable APIs that also update the datapackage!).
It looks to me that using Jinja templates to generate code is the most friendly and possibility also the most powerful way to go about this. Maybe others who have gone deep into code generation have other suggestions?
The great thing here is that, by using the SQL driver direct from the Data Package lib, we are not tied to a framework, we just generate the framework’s code. That means we can add frameworks to generate APIs for!
We are also not even tied to SQL. We currently also have a BigQuery driver, and plan to roll out others. So we potentially can roll out a combination of framework + datastore backends.
Lastly, of course, we are not tied to Python. While the lib would likely be in Python, we can generate code for any framework, and accept contributions for new frameworks as part of the codebase.
Some really good web frameworks to target would be:
- Django REST Framework
- Express.js
- Flask + SQL Alchemy + Restless
- Hapi.js
In researching this, I found GitHub - cookiecutter/cookiecutter: A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C projects. which may be of use here.
Any thoughts?