Maybe the wrong thread, but what about criteria for dropping a dataset altogether ? And/or changing / more stricter criteria ?
IMHO, the GODI cannot be used to monitor progress (because of the changing datasets, change of definition and/or criteria) over consecutive years. I think it is even mentioned somewhere in the little print, and the goal of the GODI for OKFN is probably more about pushing open data (which is laudable), but be aware that the GODI is being used / abused to monitor the progress and impact of government policy.
The yearly EU Data Portal Landscaping benchmark takes a slightly different approach (although this benchmark is more focused on policy / governance, and less on datasets itself): new criteria / requirements can - and are - added every year, but the criteria from the previous year are not removed, making it a slightly better tool to monitor progress.
I'd suggest to keep the datasets from previous years, and if the criteria for a given datasets become more strict, to add additional points.
So e.g. if the criteria for water quality are based upon availability of X and Y, that's a maximum score 20 points, and if next year you also want Z, a place still gets 20 points for X an Y, and may get an additional 10 points if Z is also available (maximum score 30).