Unable to report coverage for a single file due to import reload?

[adamjstewart]

Describe the bug

The following work fine:

pytest --cov=torchgeo.datasets tests/datasets/test_landcoverai.py
coverage run --source=torchgeo.datasets -m pytest tests/datasets/test_landcoverai.py

The following raise an error, possibly due to an import reload:

pytest --cov=torchgeo.datasets.landcoverai tests/datasets/test_landcoverai.py
coverage run --source=torchgeo.datasets.landcoverai -m pytest tests/datasets/test_landcoverai.py

Specifically, I see errors like the following:

ImportError while loading conftest '/Users/Adam/torchgeo/tests/conftest.py'.
tests/conftest.py:10: in <module>
    import torchvision
../spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torchvision/__init__.py:10: in <module>
    from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils  # usort:skip
../spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torchvision/_meta_registrations.py:25: in <module>
    @register_meta("roi_align")
../spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torchvision/_meta_registrations.py:19: in wrapper
    get_meta_lib().impl(getattr(getattr(torch.ops.torchvision, op_name), overload_name), fn)
../spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torch/library.py:322: in impl
    raise RuntimeError(
E   RuntimeError: This is not allowed since there's already a kernel registered from python overriding roi_align's behavior for Meta dispatch key and torchvision namespace.

or:

ImportError while loading conftest '/Users/Adam/torchgeo/tests/conftest.py'.
tests/conftest.py:9: in <module>
    import torch
../spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torch/__init__.py:1795: in <module>
    from torch._tensor import Tensor  # usort: skip
../spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torch/_tensor.py:21: in <module>
    from torch.overrides import (
../spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torch/overrides.py:1757: in <module>
    has_torch_function = _add_docstr(
E   RuntimeError: function '_has_torch_function' already has a docstring

I also see warnings related to reloading numpy, so I'm guessing this is related. In fact, torch does not seem to support being reloaded:

>>> from importlib import reload
>>> import torch
>>> torch = reload(torch)
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    torch = reload(torch)
  File "/Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/python-3.13.2-j7e4p3aq54ba6lpgvoyrwzbsxq3m74gl/lib/python3.13/importlib/__init__.py", line 129, in reload
    _bootstrap._exec(spec, module)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 866, in _exec
  File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/Users/Adam/spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torch/__init__.py", line 2623, in <module>
    class _TritonLibrary:
    ...<10 lines>...
            return cls.ops_table[(op_key, dispatch_key)]
  File "/Users/Adam/spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torch/__init__.py", line 2624, in _TritonLibrary
    lib = torch.library.Library("triton", "DEF")
  File "/Users/Adam/spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/torch/library.py", line 93, in __init__
    self.m: Optional[Any] = torch._C._dispatch_library(
                            ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        kind, ns, dispatch_key, filename, lineno
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
RuntimeError: Only a single TORCH_LIBRARY can be used to register the namespace triton; please put all of your definitions in a single TORCH_LIBRARY block.  If you were trying to specify implementations, consider using TORCH_LIBRARY_IMPL (which can be duplicated).  If you really intended to define operators for a single namespace in a distributed way, you can use TORCH_LIBRARY_FRAGMENT to explicitly indicate this.  Previous registration of TORCH_LIBRARY was registered at /dev/null:2623; latest registration was registered at /dev/null:2623

It seems that something, somewhere in coverage.py is trying to reload(torch) or reload(torchvision), which is not supported. I'm very curious why this only happens when reporting coverage for a single file (module) instead of a directory (package).

To Reproduce

> pip install torchgeo  # to install the dependencies
> git clone https://github.com/microsoft/torchgeo.git  # to get the tests
> cd torchgeo
> coverage run --source=torchgeo.datasets.landcoverai -m pytest tests/datasets/test_landcoverai.py

> coverage debug sys
-- sys -------------------------------------------------------
               coverage_version: 7.2.6
                coverage_module: /Users/Adam/spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages/coverage/__init__.py
                         tracer: -none-
                        CTracer: unavailable
           plugins.file_tracers: -none-
            plugins.configurers: -none-
      plugins.context_switchers: -none-
              configs_attempted: .coveragerc
                                 setup.cfg
                                 tox.ini
                                 pyproject.toml
                   configs_read: -none-
                    config_file: None
                config_contents: -none-
                      data_file: -none-
                         python: 3.13.2 (main, Feb  8 2025, 15:35:03) [Clang 16.0.0 (clang-1600.0.26.6)]
                       platform: macOS-15.3-arm64-arm-64bit-Mach-O
                 implementation: CPython
                     executable: /Users/Adam/spack/var/spack/environments/default/.spack-env/._view/xrnxxkbbchrixacht7gsgdzjxsu6eyqh/bin/python3
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 6835
                            cwd: /Users/Adam/torchgeo
                           path: /Users/Adam/spack/var/spack/environments/default/.spack-env/._view/xrnxxkbbchrixacht7gsgdzjxsu6eyqh/bin
                                 /Users/Adam/spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages
                                 /Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/python-3.13.2-j7e4p3aq54ba6lpgvoyrwzbsxq3m74gl/lib/python313.zip
                                 /Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/python-3.13.2-j7e4p3aq54ba6lpgvoyrwzbsxq3m74gl/lib/python3.13
                                 /Users/Adam/spack/opt/spack/darwin-sequoia-m2/apple-clang-16.0.0/python-3.13.2-j7e4p3aq54ba6lpgvoyrwzbsxq3m74gl/lib/python3.13/lib-dynload
                                 /Users/Adam/spack/var/spack/environments/default/.spack-env/._view/xrnxxkbbchrixacht7gsgdzjxsu6eyqh/lib/python3.13/site-packages
                    environment: HOME = /Users/Adam
                                 PYTHONPATH = /Users/Adam/spack/var/spack/environments/default/.spack-env/view/lib/python3.13/site-packages
                                 SPACK_PYTHON = /Users/Adam/spack/var/spack/environments/default/.spack-env/view/bin/python3
                   command_line: /Users/Adam/spack/var/spack/environments/default/.spack-env/view/bin/coverage debug sys
         sqlite3_sqlite_version: 3.46.0
             sqlite3_temp_store: 0
        sqlite3_compile_options: ATOMIC_INTRINSICS=1, COMPILER=clang-16.0.0, DEFAULT_AUTOVACUUM,
                                 DEFAULT_CACHE_SIZE=-2000, DEFAULT_FILE_FORMAT=4,
                                 DEFAULT_JOURNAL_SIZE_LIMIT=-1, DEFAULT_MMAP_SIZE=0, DEFAULT_PAGE_SIZE=4096,
                                 DEFAULT_PCACHE_INITSZ=20, DEFAULT_RECURSIVE_TRIGGERS,
                                 DEFAULT_SECTOR_SIZE=4096, DEFAULT_SYNCHRONOUS=2,
                                 DEFAULT_WAL_AUTOCHECKPOINT=1000, DEFAULT_WAL_SYNCHRONOUS=2,
                                 DEFAULT_WORKER_THREADS=0, DIRECT_OVERFLOW_READ, ENABLE_COLUMN_METADATA,
                                 ENABLE_FTS3, ENABLE_FTS4, ENABLE_FTS5, ENABLE_GEOPOLY,
                                 ENABLE_MATH_FUNCTIONS, ENABLE_RTREE, MALLOC_SOFT_LIMIT=1024,
                                 MAX_ATTACHED=10, MAX_COLUMN=2000, MAX_COMPOUND_SELECT=500,
                                 MAX_DEFAULT_PAGE_SIZE=8192, MAX_EXPR_DEPTH=1000, MAX_FUNCTION_ARG=127,
                                 MAX_LENGTH=1000000000, MAX_LIKE_PATTERN_LENGTH=50000,
                                 MAX_MMAP_SIZE=0x7fff0000, MAX_PAGE_COUNT=0xfffffffe, MAX_PAGE_SIZE=65536,
                                 MAX_SQL_LENGTH=1000000000, MAX_TRIGGER_DEPTH=1000,
                                 MAX_VARIABLE_NUMBER=32766, MAX_VDBE_OP=250000000, MAX_WORKER_THREADS=8,
                                 MUTEX_PTHREADS, SYSTEM_MALLOC, TEMP_STORE=1, THREADSAFE=1

> pip3 list
Package                       Version
----------------------------- ---------
aenum                         3.1.12
affine                        2.1.0
aiohttp                       3.9.5
aiosignal                     1.2.0
alabaster                     0.7.16
altgraph                      0.17.2
annotated-types               0.7.0
antlr4-python3-runtime        4.9.3
anyio                         4.0.0
appdirs                       1.4.4
appnope                       0.1.3
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arrow                         1.2.3
asttokens                     2.4.0
astunparse                    1.6.3
async-lru                     1.0.3
attrs                         23.1.0
awscli                        2.22.4
awscrt                        0.20.9
Babel                         2.15.0
beautifulsoup4                4.12.3
black                         24.3.0
bleach                        6.0.0
build                         1.2.1
cachetools                    5.2.0
Cartopy                       0.24.1
certifi                       2023.7.22
cffi                          1.17.1
cftime                        1.0.3.4
charset-normalizer            3.3.0
click                         8.1.8
click-plugins                 1.1.1
cligj                         0.7.2
cmocean                       4.0.3
colorama                      0.4.6
comm                          0.1.4
contourpy                     1.3.0
coverage                      7.2.6
cryptography                  43.0.1
cycler                        0.11.0
debugpy                       1.6.7
decorator                     5.1.1
defusedxml                    0.7.1
distro                        1.8.0
docstring_parser              0.15
docutils                      0.19
editables                     0.5
efficientnet_pytorch          0.7.1
einops                        0.8.0
et_xmlfile                    1.0.1
executing                     2.1.0
fastjsonschema                2.16.3
filelock                      3.12.4
fiona                         1.10.1
flake8                        6.1.0
fonttools                     4.39.4
fqdn                          1.5.1
frozenlist                    1.5.0
fsspec                        2024.10.0
future                        0.18.2
GDAL                          3.10.1
gdown                         5.2.0
geocube                       0.7.0
geopandas                     1.0.1
gevent                        24.11.1
greenlet                      3.1.1
h5py                          3.12.1
hatch-jupyter-builder         0.8.3
hatchling                     1.25.0
hf_transfer                   0.1.8
huggingface_hub               0.26.2
hydra-core                    1.3.2
idna                          3.4
imageio                       2.35.1
imagesize                     1.4.1
importlib_metadata            7.0.1
importlib_resources           5.12.0
iniconfig                     2.0.0
InquirerPy                    0.3.4
ipykernel                     6.29.5
ipympl                        0.9.4
ipython                       8.17.2
ipython_genutils              0.2.0
ipywidgets                    8.0.2
isoduration                   20.11.0
isort                         5.12.0
jaraco.classes                3.2.3
jedi                          0.19.2
Jinja2                        3.1.4
jmespath                      1.0.1
joblib                        1.4.2
json5                         0.9.14
jsonargparse                  4.35.0
jsonpointer                   2.0
jsonschema                    4.22.0
jsonschema-specifications     2023.12.1
jupyter_client                8.2.0
jupyter_core                  5.3.0
jupyter-events                0.10.0
jupyter-lsp                   2.2.0
jupyter_server                2.14.2
jupyter_server_terminals      0.4.4
jupyterlab                    4.0.1
jupyterlab-pygments           0.2.2
jupyterlab_server             2.27.3
jupyterlab-widgets            3.0.3
keyring                       24.3.0
kiwisolver                    1.4.5
kornia                        0.8.0
kornia_rs                     0.1.0
laspy                         2.5.4
lazy_loader                   0.4
lightly                       1.5.11
lightly_utils                 0.0.2
lightning                     2.4.0
lightning-utilities           0.11.2
lxml                          5.3.0
macholib                      1.15.2
markdown-it-py                3.0.0
MarkupSafe                    2.1.3
matplotlib                    3.10.0
matplotlib-inline             0.1.6
mccabe                        0.7.0
mdurl                         0.1.2
mistune                       2.0.5
more-itertools                9.1.0
mpmath                        1.3.0
multidict                     6.1.0
munch                         2.5.0
mypy                          1.14.1
mypy_extensions               1.0.0
nbclient                      0.6.7
nbconvert                     7.14.1
nbformat                      5.8.0
nbmake                        1.4.3
nbsphinx                      0.8.8
nest_asyncio                  1.6.0
netCDF4                       1.7.2
networkx                      3.1
notebook_shim                 0.2.3
numpy                         2.2.2
nvtx                          0.2.5
odc-geo                       0.1.2
omegaconf                     2.3.0
openpyxl                      3.1.2
overrides                     7.3.1
packaging                     24.2
pandas                        2.2.3
pandocfilters                 1.5.0
parso                         0.8.4
pathspec                      0.11.1
pexpect                       4.9.0
pfzy                          0.3.4
pillow                        11.1.0
pip                           21.2.4
pkginfo                       1.9.6
planetary-computer            0.4.9
platformdirs                  3.10.0
pluggy                        1.5.0
pretrainedmodels              0.7.4
prometheus_client             0.17.0
prompt_toolkit                3.0.38
protobuf                      3.13.0
psutil                        6.1.1
ptyprocess                    0.7.0
pure_eval                     0.2.2
pybind11                      2.13.6
pycocotools                   2.0.8
pycodestyle                   2.11.0
pycparser                     2.21
pydantic                      2.10.1
pydantic_core                 2.27.1
pyflakes                      3.1.0
Pygments                      2.18.0
pyogrio                       0.9.0
pyparsing                     3.1.2
pyproj                        3.7.0
pyproject_hooks               1.0.0
pyshp                         2.3.1
PySocks                       1.7.1
pystac                        1.11.0
pystac-client                 0.8.5
pytest                        8.2.1
pytest-cov                    4.0.0
python-dateutil               2.8.2
python-json-logger            2.0.7
pytorch-lightning             2.0.7
pytorch_sphinx_theme          0.0.24
pytz                          2024.2
PyYAML                        6.0.2
pyzmq                         26.2.0
rasterio                      1.4.3
readme_renderer               37.3
referencing                   0.35.1
requests                      2.32.3
requests-toolbelt             1.0.0
rfc3339_validator             0.1.4
rfc3986                       2.0.0
rfc3986_validator             0.1.1
rich                          13.7.1
rioxarray                     0.17.0
rpds-py                       0.20.0
Rtree                         1.3.0
ruamel.yaml                   0.17.16
ruamel.yaml.clib              0.2.12
ruff                          0.9.1
safetensors                   0.4.5
scikit-image                  0.25.1
scikit-learn                  1.6.1
scipy                         1.15.1
seaborn                       0.13.2
segmentation_models_pytorch   0.4.0
Send2Trash                    1.8.3
setuptools                    75.8.0
shapely                       2.0.7
six                           1.17.0
sniffio                       1.3.0
snowballstemmer               2.2.0
soupsieve                     2.4.1
Sphinx                        7.3.7
sphinx-copybutton             0.2.12
sphinx_design                 0.6.1
sphinx_rtd_theme              3.0.0
sphinxcontrib-applehelp       2.0.0
sphinxcontrib-devhelp         2.0.0
sphinxcontrib-htmlhelp        2.1.0
sphinxcontrib-jquery          4.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-programoutput   0.15
sphinxcontrib-qthelp          2.0.0
sphinxcontrib-serializinghtml 1.1.9
stack_data                    0.6.2
sympy                         1.13.1
terminado                     0.17.1
threadpoolctl                 3.1.0
tifffile                      2024.8.30
timm                          1.0.14
tinycss2                      1.2.1
torch                         2.6.0
torchgeo                      0.6.2
torchmetrics                  1.6.1
torchvision                   0.21.0
tornado                       6.3.3
tqdm                          4.66.3
traitlets                     5.14.3
trove-classifiers             2023.8.7
twine                         6.0.1
typeshed_client               2.1.0
typing_extensions             4.12.2
tzdata                        2023.3
uri_template                  1.2.0
urllib3                       1.26.20
wcwidth                       0.2.7
webcolors                     24.11.1
webencodings                  0.5.1
websocket-client              1.8.0
wheel                         0.41.2
widgetsnbextension            4.0.3
xarray                        2024.7.0
yarl                          1.9.2
zipp                          3.17.0
zope.event                    5.0
zope.interface                7.0.3

Expected behavior

From what I can tell, reporting coverage for a single file is supported and should work: https://discuss.python.org/t/how-to-run-coverage-for-specific-file/46019

Additional context

The pytest-cov issue seems to go away when torchgeo is not installed on the system, but the coverage invocation fails regardless. This could be an issue in pytest or torch, but I figured I would first investigate where and why the import may be reloaded.

[nedbat]

Adding some tracing code into landcoverai.py, I see this stack trace for where the module is imported:

                      <module> : /usr/local/virtualenvs/tmp-ee7baa61fc72a30/bin/coverage:8
                          main : /Users/ned/coverage/trunk/coverage/cmdline.py:970
                  command_line : /Users/ned/coverage/trunk/coverage/cmdline.py:681
                        do_run : /Users/ned/coverage/trunk/coverage/cmdline.py:855
                         start : /Users/ned/coverage/trunk/coverage/control.py:640
               _init_for_start : /Users/ned/coverage/trunk/coverage/control.py:584
                      __init__ : /Users/ned/coverage/trunk/coverage/inorout.py:269
      file_and_path_for_module : /Users/ned/coverage/trunk/coverage/inorout.py:135
                     find_spec : <frozen importlib.util>:91
                _find_and_load : <frozen importlib._bootstrap>:1360
       _find_and_load_unlocked : <frozen importlib._bootstrap>:1331
                _load_unlocked : <frozen importlib._bootstrap>:935
                   exec_module : <frozen importlib._bootstrap_external>:1026
     _call_with_frames_removed : <frozen importlib._bootstrap>:488
                      <module> : /System/Volumes/Data/src/bugs/bug1925/torchgeo/torchgeo/datasets/__init__.py:73
                _find_and_load : <frozen importlib._bootstrap>:1360
       _find_and_load_unlocked : <frozen importlib._bootstrap>:1331
                _load_unlocked : <frozen importlib._bootstrap>:935
                   exec_module : <frozen importlib._bootstrap_external>:1026
     _call_with_frames_removed : <frozen importlib._bootstrap>:488

Coverage has to find the file and path for a module. It uses importlib.find_spec to do it, which imports the module.

This happens inside a context manager that restores sys.modules, so the import will happen again when it is really supposed to. But it sounds like torch imports have side effects that aren't idempotent. I'm not sure what to do about this.