Json schema

[carlhiggs]

This pull request addresses #506, introducing

1. a JSON Schema definition (region-json-schema.json) against which we can validate our example configuration file (and later, user configuration files) based on the 2020-12 draft specification
2. a test that the example configuration matches the region json schema

Based on my local interactive tests our current example configuration file appears to validate against the region JSON schema, however the inclusion of the test is to ensure that this remains the case in our continuous integration workflow.

There is much more to do, but this will ultimately involve breaking changes to region configuration files. For example, consistent formatting of dates, and metadata for data objects.

But for now, we now have a draft JSON schema configuration file definition that validates, and this is something that we can build from. This addresses #506 in a most basic way; however, let's leave that issue open until we introduce checking of user configurations.

Checking of user configurations will require more rigorous thinking through of what are required vs optional fields. Further, changes to configuration formatting requirements will cascade to code changes (hopefully, simplifications) to accommodate these. So, this is all part of a bigger task.

[carlhiggs]

there are still some issues... that's why there are tests...

[carlhiggs]

So, the test for valid yaml schema appears to check out fine

test_0_2_schema_yaml (tests.tests.tests.test_0_2_schema_yaml)
Check if example configuration file is valid against jsonschema file. ... ok

----- but somehow unexpectedly, I think that the work around to ensure that yaml keys are treated as strings not integers although intended to only apply within the scope of the test function must be working more globally, as an error related to this occurs now in a subsequent test ---- expecting a string, but getting an int:

A couple of failing tests on the image list

ERROR: test_6_example_generate (tests.tests.tests.test_6_example_generate)

Generate resources for example region.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ghsci/process/tests/tests.py", line 132, in test_6_example_generate
    r.generate()
  File "/home/ghsci/process/subprocesses/ghsci.py", line 1028, in generate
    generate_resources(self)
  File "/home/ghsci/process/generate.py", line 114, in generate
    r.generate_report(language=language, report='indicators')
  File "/home/ghsci/process/subprocesses/ghsci.py", line 1080, in generate_report
    generate_report_for_language(
  File "/home/ghsci/process/subprocesses/_utils.py", line 211, in generate_report_for_language
    phrases = r.get_phrases(language)
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghsci/process/subprocesses/ghsci.py", line 1842, in get_phrases
    phrases[f'Image {i} file'] = city_details['images'][i]['file']
                                 ~~~~~~~~~~~~~~~~~~~~~~^^^
KeyError: 5

======================================================================
ERROR: test_8_example_generate_report_in_another_language (tests.tests.tests.test_8_example_generate_report_in_another_language)
Generate resources for example region.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ghsci/process/tests/tests.py", line 174, in test_8_example_generate_report_in_another_language
    r.generate_report('Spanish - Latin America')
  File "/home/ghsci/process/subprocesses/ghsci.py", line 1080, in generate_report
    generate_report_for_language(
  File "/home/ghsci/process/subprocesses/_utils.py", line 211, in generate_report_for_language
    phrases = r.get_phrases(language)
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ghsci/process/subprocesses/ghsci.py", line 1842, in get_phrases
    phrases[f'Image {i} file'] = city_details['images'][i]['file']
                                 ~~~~~~~~~~~~~~~~~~~~~~^^^
KeyError: 5

Not sure how to resolve this right now -- but that's the issue, can address tomorrow. Must be some simple way to specify the parsing of keys only as strings only within the scope of the schema validation test.

[shiqin-liu]

Thanks for the work Carl! As this looks like a work-in-progress to me, let's keep it i the development branch for now, rather than merging into the main. This work deserve more thinkings and discussion, circle back to the goal of replacing with Json schema, I see there are opportunities to break down based on different functional inputs, and possibly to separate the input data parameters with meta data information etc.

[carlhiggs]

Hi Shirley, Yes, it's describing the current state -- it's future work to change things. I was thinking of doing the changes incrementally, rather than all at once, Enjoy the weekend when it comes, Carl

…

On Fri, 21 Mar 2025 at 09:14, Shirley Shiqin Liu ***@***.***> wrote: Thanks for the work Carl! As this looks like a work-in-progress to me, let's keep it i the development branch for now, rather than merging into the main. This work deserve more thinkings and discussion, circle back to the goal of replacing with Json schema, I see there are opportunities to break down based on different functional inputs, and possibly to separate the input data parameters with meta data information etc. — Reply to this email directly, view it on GitHub <#526 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADDCCMXBN4TU3PLYIKWVAOL2VM4U5AVCNFSM6AAAAABZHHYLS2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONBRG44DIMJQHE> . You are receiving this because you authored the thread.Message ID: ***@***.*** com> [image: shiqin-liu]*shiqin-liu* left a comment (healthysustainablecities/global-indicators#526) <#526 (comment)> Thanks for the work Carl! As this looks like a work-in-progress to me, let's keep it i the development branch for now, rather than merging into the main. This work deserve more thinkings and discussion, circle back to the goal of replacing with Json schema, I see there are opportunities to break down based on different functional inputs, and possibly to separate the input data parameters with meta data information etc. — Reply to this email directly, view it on GitHub <#526 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADDCCMXBN4TU3PLYIKWVAOL2VM4U5AVCNFSM6AAAAABZHHYLS2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONBRG44DIMJQHE> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>