Harvesting : Fix DDI Import for otherId

[luddaniel]

What this PR does / why we need it:

This PR fixes issue while harvesting DDI with multiple otherId
This fix mirrors citation.tsv otherId configuration : allowmultiples = TRUE

dataverse/scripts/api/data/metadatablocks/citation.tsv

Line 8 in 7d4d534

otherId Other Identifier Another unique identifier for the Dataset (e.g. producer's or another repository's identifier) none 4 : FALSE FALSE TRUE FALSE FALSE FALSE citation

[coveralls]

coverage: 21.224%. remained the same
when pulling 483e99f on Recherche-Data-Gouv:fix_importDDI_otherId
into 6a00ce5 on IQSS:develop.

[landreev]

Seems non-controversial - just something we forgot to do when otherId was made multiple.
Thank you for the fix.

[ofahimIQSS]

@luddaniel Can you please provide additional details on what this PR is doing and how we can test it.

[luddaniel]

@ofahimIQSS This PR fixes the edu.harvard.iq.dataverse.util.json.JsonParseException: incorrect multiple for field otherId error when DDI harvested data contains multiple ortherId.
This can be reproduced by harvesting the following repo: https://data.progedo.fr/oai

{
  "nickName": "progedo",
  "dataverseAlias": "root",
  "type": "oai",
  "style": "default",
  "harvestUrl": "https://data.progedo.fr/oai",
  "archiveUrl": "https://data.progedo.fr",
  "archiveDescription": "This Dataset is harvested from our partners. Clicking the link will take you directly to the archival source of the data.",
  "metadataFormat": "oai_ddi25",
  "schedule": "none",
  "allowHarvestingMissingCVV": true
}

curl -H "X-Dataverse-key: 600e69df-f046-490a-824e-33f3430b9476" -H "Content-Type: application/json" -X POST "http://localhost:8080/api/harvest/clients/progedo" --upload-file "client.json"

This cannot be reproduced by Importing a Dataset into a Dataverse Installation with a DDI file as this code line is properly coded :

dataverse/src/main/java/edu/harvard/iq/dataverse/api/imports/ImportDDIServiceBean.java

Line 1439 in 7d4d534

citation.addField(FieldDTO.createMultipleCompoundFieldDTO("otherId", otherIds));

@pdurbin @landreev Is there a way to not waste too much time for this type of PR? Finding the problem and making a PR took me 1 hour whereas providing a detailed explanation, material to test took me 4 hours of headache.

[pdurbin]

@luddaniel sorry about your headache. I'm not sure what would help. Do you have any suggestions?

[ofahimIQSS]

@luddaniel Thank you for providing the testing details.

After uploading the client.json file with the same data provided, I ran the harvest job and received Success with "1660 failed" in results. I am testing in my local environment. Harvest Log/server log can be found below:

harvest_cleanup_progedo_2024-11-04T18-18-49.txt
harvest_progedo_2024-11-04T18-18-49.log
server.log

[luddaniel]

@ofahimIQSS My bad, to fix Specified metadatalanguage not allowed. I forgot to give you this instruction :

curl http://localhost:8080/api/admin/settings/:MetadataLanguages -X PUT -d '[{"locale":"en","title":"English"},{"locale":"fr","title":"Français"}]'

This should give you SUCCESS; 1072 harvested, 0 deleted, 588 failed.
Without this PR you should see edu.harvard.iq.dataverse.util.json.JsonParseException: incorrect multiple for field otherId using develop branch for example.

Updated with develop as it contained an harvesting regression fixed by #10990

[ofahimIQSS]

@luddaniel Thanks again - that did the trick. Testing Complete - Merging PR

Testing of 10772.docx