Read OME Zarr, NOT from s3 #351

will-moore · 2021-07-15T07:16:51Z

I was following the steps from https://www.youtube.com/watch?v=KposKXm7xeg, hoping to open an OME-Zarr from my local file-system, served with a simple $ python -m http.server.

The URL is http://localhost:8000/52.zarr.

There is just a single request made:

$ python -m http.server
Serving HTTP on :: port 8000 (http://[::]:8000/) ...
::ffff:127.0.0.1 - - [15/Jul/2021 08:07:47] "GET /52.zarr/?acl HTTP/1.1" 200 -

And MoBIE fails with

com.amazonaws.SdkClientException: Failed to parse XML document with handler class

Full Stack Trace

[ERROR] Module threw exception
com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$AccessControlListHandler
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:166)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseAccessControlListResponse(XmlResponsesSaxParser.java:390)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:157)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:153)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:44)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:30)
	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1726)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1446)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1368)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
	at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:3919)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1244)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1234)
	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1372)
	at org.janelia.saalfeldlab.n5.s3.N5AmazonS3Reader.<init>(N5AmazonS3Reader.java:209)
	at de.embl.cba.mobie.n5.zarr.N5S3ZarrReader.<init>(N5S3ZarrReader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader$N5S3ZarrReaderCreator.create(N5S3OMEZarrImageLoader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader.<init>(N5S3OMEZarrImageLoader.java:84)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readKey(OMEZarrS3Reader.java:42)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readURL(OMEZarrS3Reader.java:57)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3Command.openAndShow(OpenOMEZARRFromS3Command.java:39)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3AdvancedCommand.run(OpenOMEZARRFromS3AdvancedCommand.java:28)
	at org.scijava.command.CommandModule.run(CommandModule.java:196)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
	at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
	at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:152)
	... 41 more
[ERROR] Module threw exception
com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$AccessControlListHandler
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:166)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseAccessControlListResponse(XmlResponsesSaxParser.java:390)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:157)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:153)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:44)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:30)
	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1726)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1446)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1368)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
	at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:3919)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1244)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1234)
	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1372)
	at org.janelia.saalfeldlab.n5.s3.N5AmazonS3Reader.<init>(N5AmazonS3Reader.java:209)
	at de.embl.cba.mobie.n5.zarr.N5S3ZarrReader.<init>(N5S3ZarrReader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader$N5S3ZarrReaderCreator.create(N5S3OMEZarrImageLoader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader.<init>(N5S3OMEZarrImageLoader.java:84)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readKey(OMEZarrS3Reader.java:42)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readURL(OMEZarrS3Reader.java:57)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3Command.openAndShow(OpenOMEZARRFromS3Command.java:39)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3AdvancedCommand.run(OpenOMEZARRFromS3AdvancedCommand.java:28)
	at org.scijava.command.CommandModule.run(CommandModule.java:196)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
	at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
	at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:152)
	... 41 more
[ERROR] Module threw exception
com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$AccessControlListHandler
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:166)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseAccessControlListResponse(XmlResponsesSaxParser.java:390)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:157)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:153)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:44)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:30)
	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1726)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1446)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1368)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
	at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:3919)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1244)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1234)
	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1372)
	at org.janelia.saalfeldlab.n5.s3.N5AmazonS3Reader.<init>(N5AmazonS3Reader.java:209)
	at de.embl.cba.mobie.n5.zarr.N5S3ZarrReader.<init>(N5S3ZarrReader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader$N5S3ZarrReaderCreator.create(N5S3OMEZarrImageLoader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader.<init>(N5S3OMEZarrImageLoader.java:84)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readKey(OMEZarrS3Reader.java:42)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readURL(OMEZarrS3Reader.java:57)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3Command.openAndShow(OpenOMEZARRFromS3Command.java:39)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3AdvancedCommand.run(OpenOMEZARRFromS3AdvancedCommand.java:28)
	at org.scijava.command.CommandModule.run(CommandModule.java:196)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Content is not allowed in prolog.
	at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
	at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:152)
	... 41 more
[ERROR] Module threw exception
com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$AccessControlListHandler
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:166)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseAccessControlListResponse(XmlResponsesSaxParser.java:390)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:157)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:153)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:44)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:30)
	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1726)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1446)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1368)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
	at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:3919)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1244)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1234)
	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1372)
	at org.janelia.saalfeldlab.n5.s3.N5AmazonS3Reader.<init>(N5AmazonS3Reader.java:209)
	at de.embl.cba.mobie.n5.zarr.N5S3ZarrReader.<init>(N5S3ZarrReader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader$N5S3ZarrReaderCreator.create(N5S3OMEZarrImageLoader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader.<init>(N5S3OMEZarrImageLoader.java:84)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readKey(OMEZarrS3Reader.java:42)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readURL(OMEZarrS3Reader.java:57)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3Command.openAndShow(OpenOMEZARRFromS3Command.java:39)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3AdvancedCommand.run(OpenOMEZARRFromS3AdvancedCommand.java:28)
	at org.scijava.command.CommandModule.run(CommandModule.java:196)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.xml.sax.SAXParseException; lineNumber: 6; columnNumber: 3; The element type "meta" must be terminated by the matching end-tag "</meta>".
	at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
	at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
	at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:152)
	... 41 more
[ERROR] Module threw exception
java.lang.IllegalArgumentException: Endpoint does not contain a valid host name: https:///Users/wmoore
	at com.amazonaws.AmazonWebServiceClient.computeSignerByURI(AmazonWebServiceClient.java:426)
	at com.amazonaws.AmazonWebServiceClient.setEndpoint(AmazonWebServiceClient.java:318)
	at com.amazonaws.services.s3.AmazonS3Client.setEndpoint(AmazonS3Client.java:728)
	at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:453)
	at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
	at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader$N5S3ZarrReaderCreator.create(N5S3OMEZarrImageLoader.java:64)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader.<init>(N5S3OMEZarrImageLoader.java:84)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readKey(OMEZarrS3Reader.java:42)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readURL(OMEZarrS3Reader.java:57)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3Command.openAndShow(OpenOMEZARRFromS3Command.java:39)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3AdvancedCommand.run(OpenOMEZARRFromS3AdvancedCommand.java:28)
	at org.scijava.command.CommandModule.run(CommandModule.java:196)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
[ERROR] Module threw exception
com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$AccessControlListHandler
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:166)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseAccessControlListResponse(XmlResponsesSaxParser.java:390)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:157)
	at com.amazonaws.services.s3.model.transform.Unmarshallers$AccessControlListUnmarshaller.unmarshall(Unmarshallers.java:153)
	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:44)
	at com.amazonaws.services.s3.internal.ResponseHeaderHandlerChain.handle(ResponseHeaderHandlerChain.java:30)
	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:69)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1726)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleSuccessResponse(AmazonHttpClient.java:1446)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1368)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
	at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:3919)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1244)
	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1234)
	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1372)
	at org.janelia.saalfeldlab.n5.s3.N5AmazonS3Reader.<init>(N5AmazonS3Reader.java:209)
	at de.embl.cba.mobie.n5.zarr.N5S3ZarrReader.<init>(N5S3ZarrReader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader$N5S3ZarrReaderCreator.create(N5S3OMEZarrImageLoader.java:66)
	at de.embl.cba.mobie.n5.zarr.N5S3OMEZarrImageLoader.<init>(N5S3OMEZarrImageLoader.java:84)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readKey(OMEZarrS3Reader.java:42)
	at de.embl.cba.mobie.n5.zarr.OMEZarrS3Reader.readURL(OMEZarrS3Reader.java:57)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3Command.openAndShow(OpenOMEZARRFromS3Command.java:39)
	at de.embl.cba.mobie.n5.zarr.ui.OpenOMEZARRFromS3AdvancedCommand.run(OpenOMEZARRFromS3AdvancedCommand.java:28)
	at org.scijava.command.CommandModule.run(CommandModule.java:196)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.xml.sax.SAXParseException; lineNumber: 6; columnNumber: 3; The element type "meta" must be terminated by the matching end-tag "</meta>".
	at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
	at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
	at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:152)
	... 41 more

I also tried serving OME-Zarr from OMERO using https://github.com/will-moore/omero-web-zarr which failed in the same way.

Both these use-cases would be great to support.
Thanks

cc @jburel @joshmoore

The text was updated successfully, but these errors were encountered:

tischi · 2021-07-15T07:43:45Z

Indeed we did not yet support opening OME.ZARR from the file system, but @KateMoreva is working on it...
But your use-case seems different? What is the local server doing?
Serving from OMERO is also something we did not try yet.
Maybe zoom call next week such that you can explain to us those scenarios?

will-moore · 2021-07-15T08:16:02Z

I tend to serve using a minimal web server because that's what I need for vizarr and napari.
And because they work with that, they will also work with the same API provided by OMERO.

I guess that if you can read data directly from the local file-system, and you can connect to OMERO via https://github.com/stoffelc/imageJ_play_Claire then the need to support my use-case is not so urgent.

https://github.com/will-moore/omero-web-zarr is quite an early stage just now, but I like the idea that all the OME_Zarr tools could work with each other.

I'm afraid after today I'm away for a couple of weeks, but happy to continue the discussion as time allows.

constantinpape · 2021-07-15T08:32:53Z

Indeed we did not yet support opening OME.ZARR from the file system, but @KateMoreva is working on it...

This actually works already (but maybe not in the released MoBIE version).

@will-moore how are you opening the file with MoBIE? I think for what you describe it should work with Plugins->BigDataViewer->OME ZARR->Open OME ZARR from s3 ... (not that this is installed with MoBIE). Opening via MoBIE->Open MoBIE project will not work, because this expects a full MoBIE project (which has additional json metadata).

will-moore · 2021-07-15T08:41:40Z

I'm following https://www.youtube.com/watch?v=KposKXm7xeg
which does `->Open OME ZARR from s3 (Advanced)...```
What's the difference between that and the Not Advanced option?
The result appears to be the same:

If I simply point to a local zarr on my file-system, e.g. /Users/wmoore/Desktop/ZARR/data/52.zarr I get:

[ERROR] Module threw exception
java.lang.IllegalArgumentException: Endpoint does not contain a valid host name: https:///Users/wmoore
	at com.amazonaws.AmazonWebServiceClient.computeSignerByURI(AmazonWebServiceClient.java:426)
	at com.amazonaws.AmazonWebServiceClient.setEndpoint(AmazonWebServiceClient.java:318)

will-moore · 2021-07-15T08:43:19Z

A screenshot is sometimes handy:

tischi · 2021-07-15T09:36:33Z

There's no significant difference between advanced and the other option.
It simply (as the name suggests) only opens data from S3.

constantinpape · 2021-07-15T09:37:07Z

Ok, that's the right way to do it, @will-moore and your example is something we want to support.
Looking at your stack trace, I think that the issue is that some of our parser logic assumes an s3 url, so it fails for the local server.
We should fix this.

KateMoreva · 2021-07-15T09:37:36Z

I'm following https://www.youtube.com/watch?v=KposKXm7xeg
which does `->Open OME ZARR from s3 (Advanced)...```
What's the difference between that and the Not Advanced option?
The result appears to be the same:

If I simply point to a local zarr on my file-system, e.g. /Users/wmoore/Desktop/ZARR/data/52.zarr I get:
[ERROR] Module threw exception
java.lang.IllegalArgumentException: Endpoint does not contain a valid host name: https:///Users/wmoore
	at com.amazonaws.AmazonWebServiceClient.computeSignerByURI(AmazonWebServiceClient.java:426)
	at com.amazonaws.AmazonWebServiceClient.setEndpoint(AmazonWebServiceClient.java:318)

For opening ome.zarr files directly from the file system you need to use Plugins>BigDataViewer>OME ZARR>Open OME ZARR from the file system... command, which is in not released version yet.

tischi · 2022-06-04T09:07:07Z

@will-moore
This is in MoBIE now: Plugins › BigDataViewer › OME-Zarr › Open OME-Zarr From File System...
Does that work for you?

will-moore · 2022-06-08T15:02:25Z

Hi, @tischi I just re-installed Fiji on my new Mac M1 and added the MoBIE update site...
But I don't see the OME-Zarr option under BigDataViewer:

Am I missing some other plugin I need?

tischi · 2022-06-08T15:16:10Z

Weird, I see it.
Are you sure you enabled the MoBIE update site and restarted Fiji?

will-moore · 2022-06-08T15:58:19Z

Ah - thanks, sorry, I thought I had restarted, but anyway - another restart fixed it...

I'm looking at a local version of https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.2/6001240.zarr and I just see black.
The local copy looks fine in napari so I think the data is OK. I'm just not familiar with how to adjust contrast etc.

When I open https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.2/6001240.zarr as remote s3 I just see the labels, not the other channels.
I press S to open the brightness and color controls, but I can't see how to adjust any channel other than the labels?

Trying https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.3/9836842.zarr (no labels) I only see a black screen - adjusting contrast doesn't seem to help:

Thanks!

tischi · 2022-06-08T16:40:48Z

You have to press P to bring up the new B&C UI.

constantinpape · 2022-06-09T08:49:39Z

Opening from S3 and then adjusting the colorlimits via P works for me:

will-moore · 2022-06-09T09:00:42Z

Ah - great, it's working for me on local data!

But I didn't see the P command in the help doc that is shown when you press F1?
Thanks!

constantinpape · 2022-06-09T09:02:05Z

But I didn't see the P command in the help doc that is shown when you press F1?

That should probably be raised as an issue in https://github.com/bigdataviewer/bigdataviewer-core.

will-moore · 2022-06-09T09:14:42Z

Added comment to bigdataviewer/bigdataviewer-core#64

constantinpape · 2022-06-09T09:17:28Z

Thanks @will-moore. I think everything is working as expected here now; I am closing it; feel free to reopen if something else comes up.

will-moore mentioned this issue Nov 2, 2021

document client feature support ome/ngff#71

Closed

will-moore mentioned this issue Jun 9, 2022

Amend Help bigdataviewer/bigdataviewer-core#64

Open

constantinpape closed this as completed Jun 9, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Read OME Zarr, NOT from s3 #351

Read OME Zarr, NOT from s3 #351

will-moore commented Jul 15, 2021

tischi commented Jul 15, 2021

will-moore commented Jul 15, 2021

constantinpape commented Jul 15, 2021

will-moore commented Jul 15, 2021

will-moore commented Jul 15, 2021

tischi commented Jul 15, 2021

constantinpape commented Jul 15, 2021

KateMoreva commented Jul 15, 2021

tischi commented Jun 4, 2022

will-moore commented Jun 8, 2022

tischi commented Jun 8, 2022

will-moore commented Jun 8, 2022 •

edited

Loading

tischi commented Jun 8, 2022

constantinpape commented Jun 9, 2022

will-moore commented Jun 9, 2022

constantinpape commented Jun 9, 2022

will-moore commented Jun 9, 2022

constantinpape commented Jun 9, 2022

Read OME Zarr, NOT from s3 #351

Read OME Zarr, NOT from s3 #351

Comments

will-moore commented Jul 15, 2021

tischi commented Jul 15, 2021

will-moore commented Jul 15, 2021

constantinpape commented Jul 15, 2021

will-moore commented Jul 15, 2021

will-moore commented Jul 15, 2021

tischi commented Jul 15, 2021

constantinpape commented Jul 15, 2021

KateMoreva commented Jul 15, 2021

tischi commented Jun 4, 2022

will-moore commented Jun 8, 2022

tischi commented Jun 8, 2022

will-moore commented Jun 8, 2022 • edited Loading

tischi commented Jun 8, 2022

constantinpape commented Jun 9, 2022

will-moore commented Jun 9, 2022

constantinpape commented Jun 9, 2022

will-moore commented Jun 9, 2022

constantinpape commented Jun 9, 2022

will-moore commented Jun 8, 2022 •

edited

Loading