Cannot Download PCAM dataset for val split

[chrisyxue]

🐛 Describe the bug

import torchvision.datasets as datasets
transform_train = transforms.Compose([
            transforms.ColorJitter(),
            transforms.RandomHorizontalFlip(),
            transforms.RandomVerticalFlip(),
            transforms.Resize(args.input_size),
            transforms.ToTensor(),
            transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) 
        ]) 

dataset_train = datasets.PCAM(
    root = args.data_path,
    split = 'val',
    transform = transform_train,
    download=True
)

The error is like:

RuntimeError: The MD5 checksum of the download file /localscratch2/xuezhiyu/datasets/pcam/camelyonpatch_level_2_split_valid_x.h5.gz does not match the one on record.Please delete the file and try again. If the issue persists, please report this to torchvision at https://github.com/pytorch/vision/issues.

Versions

Collecting environment information...
PyTorch version: 1.12.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.8 (default, Apr 13 2021, 19:58:26) [GCC 7.3.0] (64-bit runtime)
Python platform: Linux-5.13.0-27-generic-x86_64-with-glibc2.10
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration:
GPU 0: NVIDIA RTX A5000
GPU 1: NVIDIA RTX A5000
GPU 2: NVIDIA RTX A5000
GPU 3: NVIDIA RTX A5000

Nvidia driver version: 470.57.02
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.3.2
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.3.2
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.8.0.5
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.0.5
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.0.5
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.0.5
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.0.5
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.0.5
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.0.5
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn.so.8.3.1
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.3.1
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.3.1
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.3.1
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.3.1
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.3.1
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.3.1
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn.so.8.2.4
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.2.4
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.2.4
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.2.4
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.2.4
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.2.4
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.2.4
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.20.1
[pip3] numpydoc==1.1.0
[pip3] torch==1.12.1+cu113
[pip3] torchaudio==0.12.1+cu113
[pip3] torchmultimodal==0.1.0a0
[pip3] torchvision==0.13.1+cu113
[conda] blas 1.0 mkl
[conda] mkl 2021.2.0 h06a4308_296
[conda] mkl-service 2.3.0 py38h27cfd23_1
[conda] mkl_fft 1.3.0 py38h42c9631_2
[conda] mkl_random 1.2.1 py38ha9443f7_2
[conda] numpy 1.20.1 py38h93e21f0_0
[conda] numpy-base 1.20.1 py38h7d8b39e_0
[conda] numpydoc 1.1.0 pyhd3eb1b0_1
[conda] torch 1.12.1+cu113 pypi_0 pypi
[conda] torchaudio 0.12.1+cu113 pypi_0 pypi
[conda] torchmultimodal 0.1.0a0 dev_0
[conda] torchvision 0.13.1+cu113 pypi_0 pypi

cc @pmeier

[pmeier]

Can confirm:

from torchvision.datasets.utils import download_file_from_google_drive

download_file_from_google_drive(
    file_id="1hgshYGWK8V-eGRy8LToWJJgDU_rXWVJ3",
    root=".",
    filename="camelyonpatch_level_2_split_valid_x.h5",
    md5="d8c2d60d490dbd479f8199bdfa0cf6ec",
)

Checksum of the downloaded file is d5b63470df7cfa627aeec8b9dc0c066e. The one we have on record is on par with what the author provided.

[pmeier]

The checksums for the "val" and "test" split are swapped:

d8c2d60d490dbd479f8199bdfa0cf6ec  camelyonpatch_level_2_split_test_x.h5.gz
60a7035772fbdb7f34eb86d4420cf66a  camelyonpatch_level_2_split_test_y.h5.gz
d5b63470df7cfa627aeec8b9dc0c066e  camelyonpatch_level_2_split_valid_x.h5.gz
2b85f58b927af9964a4c15b8f7e8f179  camelyonpatch_level_2_split_valid_y.h5.gz

vision/torchvision/datasets/pcam.py

Lines 43 to 66 in 7046e56

	"test": {
	"images": (
	"camelyonpatch_level_2_split_test_x.h5",
	"1qV65ZqZvWzuIVthK8eVDhIwrbnsJdbg_",
	"d5b63470df7cfa627aeec8b9dc0c066e",
	),
	"targets": (
	"camelyonpatch_level_2_split_test_y.h5",
	"17BHrSrwWKjYsOgTMmoqrIjDy6Fa2o_gP",
	"2b85f58b927af9964a4c15b8f7e8f179",
	),
	},
	"val": {
	"images": (
	"camelyonpatch_level_2_split_valid_x.h5",
	"1hgshYGWK8V-eGRy8LToWJJgDU_rXWVJ3",
	"d8c2d60d490dbd479f8199bdfa0cf6ec",
	),
	"targets": (
	"camelyonpatch_level_2_split_valid_y.h5",
	"1bH8ZRbhSVAhScTS0p9-ZzGnX91cHT3uO",
	"60a7035772fbdb7f34eb86d4420cf66a",
	),
	},

The same is true for the checksums provided by the author.

[pmeier]

See basveeling/pcam#4.