Bug report: Json API doesn't allow round-trip of license information

[sergejzr]

Title updated. In discussion this issue we discovered that the root cause is a bug/mismatch between the JSON export and import format for licenses. A resolution to this issue should address that. (Note there is a workaround.)

Overview of the Feature Request

Dear Dataverse developers,

Unfortunately, I could not find a possibility to attach Terms to a dataset using API.

What kind of user is the feature intended for?
(Example users roles: API User, Curator, Depositor, Guest, Superuser, Sysadmin)
API User

What inspired the request?

There is a section in den docs explaining how to manage the tems dataverse-wide[1], however not how to attach them to datasets.
[1] https://guides.dataverse.org/en/latest/api/native-api.html#id202

What existing behavior do you want changed?
When I try to do it the same way that dataverse respond, the data is getting accepted, however there is no result and the dataset is created with "custom terms"

 {
        "metadataLanguage": "en",
        "datasetVersion": {
            "license": {
                "name": "CC0 1.0",
                "uri": "http://creativecommons.org/publicdomain/zero/1.0"
            },
            "metadataBlocks": {.......}
        }
    }

I would expect that this json would change the license to CC0

Any brand new behavior do you want to add to Dataverse?

Any related open or closed issues to this feature request?
All issues about Terms are somehow related, I guess..
Thank you very much
Sergej

[qqmyers]

The example at https://guides.dataverse.org/en/latest/api/native-api.html#import-a-dataset-into-a-dataverse-collection shows adding a license. Are you using that API call or something else?

The Admin API for managing licenses allows new licenses to be added to the set that are allowed. If you try importing a dataset with a license not in that list, it would be an error.

Also, FWIW, the Semantic API, https://guides.dataverse.org/en/latest/developers/dataset-semantic-metadata-api.html, allows setting other fields on the terms of use and access pane. It's the only API for those fields afaik.

[sergejzr]

Thanks!

The example at https://guides.dataverse.org/en/latest/api/native-api.html#import-a-dataset-into-a-dataverse-collection shows adding a license. Are you using that API call or something else?

I am using exactly that. The call is also accepted without errors, however, instead of CC0, the undefined terms are shown:

The Admin API for managing licenses allows new licenses to be added to the set that are allowed. If you try importing a dataset with a license not in that list, it would be an error.

Yes, I checked the terms. Indeed, this field is returned when accessing a dataset through API. There is no Error and no result unfortunately..

Also, FWIW, the Semantic API, https://guides.dataverse.org/en/latest/developers/dataset-semantic-metadata-api.html, allows setting other fields on the terms of use and access pane. It's the only API for those fields afaik.
I checked it, however did not find the way yet. I will keep trying..

Thank you very much for the hint
Sergej

[pdurbin]

Possibly related:

• Ensure that datasets published using the Dataverse APIs include a license or have Terms of Use #9148

[qqmyers]

FWIW: It looks like this is due to a change in the json output that wasn't reflected in the create API (but is in the documentation) - the parsing currently assumes that "license" is a string that is the name or uri, not a json object with both.

So - the workaround would be to add something like "license":"CC BY 4.0" in your dataset json.

The fact that the create/import APIs don't handle the same format as the output and that sending an object for the license doesn't report an error is a bug that can get queued up.

[sergejzr]

Thanks! Yes, just putting the name works for now as a workaround.

[jggautier]

I've run into this issue a few times, too, as I try to help users use the "Create a Dataset in a Collection" API endpoint and forget that following the example in the documentation (the dataset-create-new-all-default-fields.json file) won't work.

It sounds like we'd prefer that the API endpoint expect that the value for the "license" key be an object with both the license name and URI, to match how the license metadata is exported in the Dataverse JSON format. And that the endpoint returns an error when the value for the "license" key is anything other than an object with the license name and URI.

Can we close this issue and:

• Open a new issue to propose that solution?
• Open a second issue that updates the example in the API documentation, assuming that we can more quickly update the documentation to reflect what works now? If we agree, non-developer me would even volunteer to submit a PR to edit that dataset-create-new-all-default-fields.json file. :)

[landreev]

This is how the license is exported:

bld.add("license", jsonObjectBuilder()
     .add("name", DatasetUtil.getLicenseName(dsv))
     .add("uri", DatasetUtil.getLicenseURI(dsv)));

and in the DatasetUtil:

public static String getLicenseName(DatasetVersion dsv) {
    License license = DatasetUtil.getLicense(dsv);
    return license != null ? getLocalizedLicenseDetails(license,"NAME") : BundleUtil.getStringFromBundle("license.custom");
}

But in json parser the license is looked up by the name provided, i.e. it must match the name in the license table.

[landreev]

This is where the different names are specified, for CC-BY 4.0 specifically:
the json we distribute for the license:

% cat ./scripts/api/data/licenses/licenseCC-BY-4.0.json
{
  "name": "CC BY 4.0",
  "uri": "http://creativecommons.org/licenses/by/4.0",
  "shortDescription": "Creative Commons Attribution 4.0 International License.",
  "iconUrl": "https://licensebuttons.net/l/by/4.0/88x31.png",
  "active": true,
  "sortOrder": 2
}

but for display and export the name from the properties bundle is used:

% grep 'BY 4.0' ./src/main/java/propertyFiles/License.properties 
license.cc_by_4.0.name=CC-BY 4.0

It's NOT just the different names though. The import appears to only accept the name as a primitive string, as in "license" : "...". Providing it in the exported format, as "license" : {"name":"...","uri":"..."}, even with the correct name, appears to fail.

[landreev]

I'm fixing this bug as part of #9148; just put together a prototype fix.
Does anyone here have a strong opinion on whether I should leave some backward compatibility in place?
Specifically, with the API accepting the license info as exported, for ex., as

"license": {
   "name": "CC0 1.0",
   "uri": "http://creativecommons.org/publicdomain/zero/1.0"
}

should it still accept the current

"license" : "CC0 1.0"

as well?

[pdurbin]

Yes, I think it would be nice to accept both forms. Backward compatibility is important.

[landreev]

I was leaning towards leaving backward compatibility in place as well. Just checked it in (but, work in progress).

[pdurbin]

To quote Rich Hickey:

“They can say, “I am in Bermuda this week, but next week I will try foo2. That sounds awesome.” But right now, my web service is going to keep working, because it calls foo, and you did not take it away from me while I was on vacation.”

From https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md

[pdurbin]

Closing manually because this PR was merged even though it says closed:

• 9148 Ensure that license/terms are set via API (create/update/publish) #9568