-
Notifications
You must be signed in to change notification settings - Fork 741
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Image /mnt overriding my machine's /mnt causing errors #530
Comments
Hi @japhill , I do see a /mnt directory:
which is empty, so I think your suggestion of something like "RUN rm -rf /mnt/" makes sense. I'll also do a quick search to see if there are better approaches here. Thanks for the feedback. I'll track internally and make sure this is updated in the next release. |
@japhill But for now, I also wonder if you can use the |
Just to add to the previous post. Your file system is not accessible from the container. Using -B option you may map (mount) some directories from your file system to be accessible from within the container. There is a similar issue with the similar content. #506 (comment) |
Thank you for the help. I'm somewhat new to Singularity and how it works, so this has been helpful. |
Hi @japhill , to give you an update, I plan to add this section to our FAQ in the next release: Issues with
|
Thank you very much. That will help out the folks new to docker/singularity. After using the -B option, we are good to go now!
From: Pi-Chuan Chang ***@***.***>
Sent: Friday, March 25, 2022 7:19 PM
To: google/deepvariant ***@***.***>
Cc: Jason Phillips ***@***.***>; Mention ***@***.***>
Subject: Re: [google/deepvariant] Image /mnt overriding my machine's /mnt causing errors (Issue #530)
Hi @japhill<https://github.com/japhill> , to give you an update, I plan to add this section to our FAQ in the next release:
…________________________________
Issues with /mnt/
User reported that sometimes their setup uses /mnt/, which exists in our Docker image, and it has caused an issue in Singularity.
You can use -B in Singularity to avoid this issue. See:
#530 (comment)<#530 (comment)> for more details.
________________________________
Given that this solution works, and is a standard Singularity flag, I won't plan to remove /mnt from our Docker images in the future.
—
Reply to this email directly, view it on GitHub<#530 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHAS6GHJOPUE7DQPYAT264TVBZCWLANCNFSM5Q7A6FXQ>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
I ran this using singularity. I tried to tell the system to read and write files to a folder in my machine's /mnt/ folder. I keep getting an error. After inspecting, it looks like this image has an empty /mnt/ directory that is not writable. This is a problem for us and many users because it is very common to store large amounts of data in the /mnt/ folder on servers that access shared space from a common storage device. Please tell your Dockerfile to "RUN rm -rf /mnt/" or something (I'm not a docker expert by any means). The deepvariant docker container clearly does not need /mnt/
Setup
Steps to reproduce:
...please note that /mnt/share is an NFS mount. My server mounts a drive on another machine running nfs.service
mkdir -p /mnt/share/jasontest
cd /mnt/share/jasontest
INPUT_DIR="${PWD}/quickstart-testdata"
DATA_HTTP_DIR="https://storage.googleapis.com/deepvariant/quickstart-testdata"
mkdir -p ${INPUT_DIR}
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/NA12878_S1.chr20.10_10p1mb.bam
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/NA12878_S1.chr20.10_10p1mb.bam.bai
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/test_nist.b37_chr20_100kbp_at_10mb.bed
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/test_nist.b37_chr20_100kbp_at_10mb.vcf.gz
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/test_nist.b37_chr20_100kbp_at_10mb.vcf.gz.tbi
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/ucsc.hg19.chr20.unittest.fasta
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/ucsc.hg19.chr20.unittest.fasta.fai
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/ucsc.hg19.chr20.unittest.fasta.gz
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/ucsc.hg19.chr20.unittest.fasta.gz.fai
wget -P ${INPUT_DIR} "${DATA_HTTP_DIR}"/ucsc.hg19.chr20.unittest.fasta.gz.gzi
BIN_VERSION="1.3.0"
OUTPUT_DIR="${PWD}/quickstart-output"
mkdir -p "${OUTPUT_DIR}"
singularity run -B /usr/lib/locale/:/usr/lib/locale/ docker://google/deepvariant:"${BIN_VERSION}" /opt/deepvariant/bin/run_deepvariant --model_type=WGS --ref="${INPUT_DIR}"/ucsc.hg19.chr20.unittest.fasta --reads="${INPUT_DIR}"/NA12878_S1.chr20.10_10p1mb.bam --regions "chr20:10,000,000-10,010,000" --output_vcf="${OUTPUT_DIR}"/output.vcf.gz --output_gvcf="${OUTPUT_DIR}"/output.g.vcf.gz --intermediate_results_dir "${OUTPUT_DIR}/intermediate_results_dir" --num_shards=1
stack trace:
I0317 09:40:21.184321 140398386173760 run_deepvariant.py:341] Creating a directory for intermediate results in /mnt/share/jasontest/quickstart-output/intermediate_results_dir
Traceback (most recent call last):
File "/opt/deepvariant/bin/run_deepvariant.py", line 493, in
app.run(main)
File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "/opt/deepvariant/bin/run_deepvariant.py", line 460, in main
intermediate_results_dir = check_or_create_intermediate_results_dir(
File "/opt/deepvariant/bin/run_deepvariant.py", line 343, in check_or_create_intermediate_results_dir
os.makedirs(intermediate_results_dir)
File "/usr/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/usr/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/usr/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 1 more time]
File "/usr/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/mnt/share'
Does the quick start test work on your system?
The quick test works on my system as long as my data is not in the /mnt/ folder.
Any additional context:
The text was updated successfully, but these errors were encountered: