Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test nix with the Issue1049-numpy_2.0.0_breaks_things branch #1052

Closed
wants to merge 55 commits into from
Closed

Test nix with the Issue1049-numpy_2.0.0_breaks_things branch #1052

wants to merge 55 commits into from

Conversation

wd15
Copy link
Contributor

@wd15 wd15 commented Jun 21, 2024

No description provided.

guyer and others added 23 commits June 18, 2024 09:04
@guyer
Monkeypatch versioneer.py
72b3885 
- Python 3.12 doesn't like SafeConfigParser
- Python 2.7 doesn't like versioneer > 0.18
@guyer
Replace obj2sctype
2c7136d
@guyer
Replace .getsctype() with .dtype
2eed418
@guyer
Replace NUMERIX.issubclass_() with issubclass()
62bd0ff
@guyer
Replace itemset() with indexing
4e7923a
@guyer
Fix obsolete name for infinity
9921bf8
@guyer
Fix comparison operator representation
55f78f3
@guyer
Use ins.argrepr for representation of operator variables
0b14ced
@guyer
Remove future.standard_library
08124c2 
This library no longer exists in Python 3.12

It looks like it isn't even needed in Python 2.7 to use pickle
and redirecting_html.py should go away (but fixed it anyway).
@guyer
Update locks
d91ccdc
@guyer
Add return_scalar to __array_wrap__ for NumPy 2.0
29047ee
@guyer
Add copy= argument to __array__ for NumPy 2.0
69a4e24
@guyer
Address copy=None for NumPy < 2.0
908825d 
Based on https://github.com/scipy/scipy/pull/20172/files
as suggested by https://numpy.org/devdocs/numpy_2_0_migration_guide.html#adapting-to-changes-in-the-copy-keyword
@guyer
Defer return_scalar for OperatorVariables
6e6c68e
@guyer
Fix polarity of floating CellVariable check
729e2c5 
Broken in 2eed418
@guyer
Replace deprecated numpy.in1d with numpy.isin
130983f
@guyer
Replace deprecated tol= with rtol= to SciPy Krylov solvers
afd1697
@guyer
Replace deprecated atol='legacy' for SciPy Krylov solves
43c143a
@guyer
Remove internal implementation details of numpy.array2string
3a1e316
@guyer
Override legacy formatting set for tests
0ce2a46
@guyer
Lock out petsc 3.21
f7c0ada 
petsc(4py) 3.21.2 crashes when running tests in parallel:

```
mesh2D (examples.cahnHilliard)
Doctest: examples.cahnHilliard.mesh2D ...  Internal error 1 in DMUMPS_LOAD_RECV_MSGS           1
 Internal error 1 in DMUMPS_LOAD_RECV_MSGS           1
 Internal error 2 in DMUMPS_LOAD_RECV_MSGS  1275068685          52
 Internal error 1 in DMUMPS_LOAD_PROCESS_MESSAGE
 Internal error 1 in DMUMPS_LOAD_RECV_MSGS           1
Abort(-99) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 0
Abort(-99) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 1
Abort(-99) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 1
Abort(-99) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 0
Abort(-99) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 0
Abort(-99) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 0
Abort(-99) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 1
Abort(-99) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 1
Abort(-99) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 1
Abort(-99) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 0
Abort(-99) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 1
Abort(-99) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 1
Abort(-99) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 1
Abort(-99) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 0
Abort(-99) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -99) - process 0
 Internal error 1 in DMUMPS_LOAD_PROCESS_MESSAGE
 Internal error 1 in DMUMPS_LOAD_RECV_MSGS           1
 Internal error 2 in DMUMPS_LOAD_RECV_MSGS  1275068685          52
 Internal error 1 in DMUMPS_LOAD_PROCESS_MESSAGE
 Internal error 1 in DMUMPS_LOAD_RECV_MSGS           1
 Internal error 1 in DMUMPS_LOAD_PROCESS_MESSAGE
 Internal error 1 in DMUMPS_LOAD_RECV_MSGS           1
 Internal error 2 in DMUMPS_LOAD_RECV_MSGS  1275068685          52
 Internal error 1 in DMUMPS_LOAD_PROCESS_MESSAGE
 Internal error 1 in DMUMPS_LOAD_RECV_MSGS           1
```
@guyer
Make directory that already exists
b9f75b8
@wd15
Update nixos to 24.05
a7721ca
@wd15 wd15 mentioned this pull request Jun 21, 2024
Merged
@guyer
Copy link
Member

guyer commented Jun 24, 2024

@wd15: Having tried a bunch of diagnostics and getting nothing to show up in the nix tests, I think what's happening is that:

  • your nix installation already has FiPy in it?
  • nix develop --command bash -c "python setup.py test 2> test.out" || true runs the fipy tests in the source directory, but when the tests import fipy, they don't import from the source directory, they import the fipy that's already installed.

This would explain why we see changes in the doctests (because it's extracting those from the source code), but we don't see changes to the code (because it's running the code that's already installed).

My naive attempt to do something about that (2dd8321) results in

× This environment is externally managed
╰─> This command has been disabled as it tries to modify the immutable
    `/nix/store` filesystem.
    
    To use Python with Nix and nixpkgs, have a look at the online documentation:
    <https://nixos.org/manual/nixpkgs/stable/#python>.

I don't (want to) understand nix well enough to work around that.

@wd15
Copy link
Contributor Author

wd15 commented Jun 24, 2024

It is definitely using the updated version of fipy/tests/doctestPlus.py as it prints on at the top of the file and prints to the locally cloned version. Installing with pip install won't work as Nix doesn't allow this for obvious reasons. Although it's using the cloned version of fipy/tests/doctestPlus.py, it is never failing with the raise Exception in the testmod function. What is going on?

@wd15
Copy link
Contributor Author

wd15 commented Jun 24, 2024

Ok, although fipy/tests/doctestPlus.py is imported, the debugging version of fipy/tests/test.py is never imported. Why is that?

@wd15
Copy link
Contributor Author

wd15 commented Jun 24, 2024 •
edited
Loading

Other than when fipy uses fipy.test(), fipy.tests.test is never imported in the fipy source. It's handed over to setup.py which knows what to do. It's from this file that we're not getting the debugging info that we should be printing and thus the change in the way numpy operates.

@guyer
Copy link
Member

guyer commented Jun 24, 2024

Let me do some experiments

@wd15
Copy link
Contributor Author

wd15 commented Jun 24, 2024

Oh! Building the egg info actually made it use the modified test module.

@guyer
Copy link
Member

guyer commented Jun 24, 2024

Oh! Building the egg info actually made it use the modified test module.

Yes. Turns out it isn't a nix thing, per se. Fresh checkout, without building egg_info, doesn't execute the relevant tweaks to printoptions. After building egg_info, for any reason, all is OK.

Still not right for this particular Action; it doesn't appear to be running the tests after it builds egg_info. I'll leave that to you.

@wd15
Copy link
Contributor Author

wd15 commented Jun 24, 2024

Oh! Building the egg info actually made it use the modified test module.

Yes. Turns out it isn't a nix thing, per se. Fresh checkout, without building egg_info, doesn't execute the relevant tweaks to printoptions. After building egg_info, for any reason, all is OK.

Still not right for this particular Action; it doesn't appear to be running the tests after it builds egg_info. I'll leave that to you.

Very good. Will get this cleaned up.

@wd15
Copy link
Contributor Author

wd15 commented Jun 24, 2024

Just one failure now, which also occurs locally as well. Some sort of reordering error with Scipy / Numpy interaction.

@wd15 wd15 closed this Jun 25, 2024
@wd15 wd15 deleted the issue1049-Numpy_2.0.0_breaks_things-nix branch June 25, 2024 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@wd15 @guyer