Updating to ray2 #1982

joshua-cogliati-inl · 2022-09-29T15:54:00Z

Pull Request Description

What issue does this change request address?

#1972

What are the significant changes in functionality due to this change request?

Updates ray to version 2
Always pass PYTHONPATH to ray init.

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

1. Review all computer code.
2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
8. If an analytic test is changed/added is the the analytic documentation updated/added?
9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

moosebuild · 2022-09-30T15:57:43Z

Job Precheck on 743a8f0 : invalidated by @joshua-cogliati-inl

failed in fetch

moosebuild · 2022-09-30T16:27:53Z

Job Precheck on 743a8f0 : invalidated by @joshua-cogliati-inl

failed in fetch

moosebuild · 2022-09-30T16:43:25Z

Job Precheck on 743a8f0 : invalidated by @milljm

moosebuild · 2022-09-30T17:18:21Z

Job Precheck on 743a8f0 : invalidated by @joshua-cogliati-inl

failed in fetch

moosebuild · 2022-10-03T14:12:21Z

Job Test qsubs sawtooth on ca0493d : invalidated by @joshua-cogliati-inl

FAILED: Diff tests/cluster_tests/AdaptiveSobol/test_parallel_adaptive_sobol

joshua-cogliati-inl · 2022-10-03T14:15:31Z

Hm, mac failed with:

(    0.14 sec) Job Handler              : DEBUG           -> Initializing ray locally with num_cpus:  4
2022-09-30 14:55:25,799	ERROR node.py:742 -- Unable to succeed in selecting a random port.
Traceback (most recent call last):
  File "/Users/civet/civet/build_0/raven/raven_framework.py", line 26, in <module>
    sys.exit(main(True))
  File "/Users/civet/civet/build_0/raven/ravenframework/Driver.py", line 203, in main
    raven()
  File "/Users/civet/civet/build_0/raven/ravenframework/Driver.py", line 155, in raven
    simulation.initialize()
  File "/Users/civet/civet/build_0/raven/ravenframework/Simulation.py", line 543, in initialize
    self.jobHandler.initialize()
  File "/Users/civet/civet/build_0/raven/ravenframework/JobHandler.py", line 140, in initialize
    self.__initializeRay()
  File "/Users/civet/civet/build_0/raven/ravenframework/JobHandler.py", line 224, in __initializeRay
    self.rayServer = ray.init(num_cpus=int(self.runInfoDict['totalNumCoresUsed']),include_dashboard=db) if _rayAvail else \
  File "/Users/civet/.conda/envs/raven_libraries/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/Users/civet/.conda/envs/raven_libraries/lib/python3.8/site-packages/ray/_private/worker.py", line 1420, in init
    _global_node = ray._private.node.Node(
  File "/Users/civet/.conda/envs/raven_libraries/lib/python3.8/site-packages/ray/_private/node.py", line 267, in __init__
    self._ray_params.update_pre_selected_port()
  File "/Users/civet/.conda/envs/raven_libraries/lib/python3.8/site-packages/ray/_private/parameter.py", line 326, in update_pre_selected_port
    raise ValueError(
ValueError: Ray component dashboard_agent_grpc is trying to use a port number 65534 that is used by other components.
Port information: {'gcs': 'random', 'object_manager': 'random', 'node_manager': 'random', 'gcs_server': 65534, 'client_server': 'random', 'dashboard': 'random', 'dashboard_agent_grpc': 65534, 'dashboard_agent_http': 52365, 'metrics_export': 65535, 'redis_shards': 'random', 'worker_ports': 'random'}
If you allocate ports, please make sure the same port is not used by multiple components.

Running test failed with exit code -15
(678F1/819) Failed (  7.69sec)tests/framework/InternalParallelTests/ROMscikit

moosebuild · 2022-10-03T14:15:53Z

Job Test mac on ca0493d : invalidated by @joshua-cogliati-inl

ValueError: Ray component dashboard_agent_grpc is trying to use a port number 65534 that is used by other components.

moosebuild · 2022-10-03T15:56:10Z

Job Test Ubuntu 16 on 743a8f0 : invalidated by @joshua-cogliati-inl

This is a \test\for\Everything

moosebuild · 2022-10-03T15:58:54Z

Job Test Ubuntu 16 on 743a8f0 : canceled by @joshua-cogliati-inl

f: \ stuff

moosebuild · 2022-10-03T16:00:52Z

Job Test Ubuntu 16 on 743a8f0 : invalidated by @joshua-cogliati-inl

Restarting.

moosebuild · 2022-10-04T18:00:06Z

Job Test Ubuntu 16 on ca0493d : invalidated by @joshua-cogliati-inl

restarted civet

wangcj05 · 2022-10-05T21:39:05Z

Windows failed.

moosebuild · 2022-10-05T22:51:39Z

Job Test Fedora 31 on 6bbda33 : invalidated by @joshua-cogliati-inl

FAILED: Diff tests/framework/PostProcessors/EconomicRatio/timeDepDataset

moosebuild · 2022-11-15T18:17:08Z

Job Mingw Test on 6bbda33 : invalidated by @wangcj05

testing

moosebuild · 2022-11-21T18:09:20Z

Job Test Ubuntu 18 PIP on 164e372 : invalidated by @joshua-cogliati-inl

restarted civet

moosebuild · 2022-11-21T18:13:13Z

Job Test Ubuntu 18 PIP on 164e372 : invalidated by @joshua-cogliati-inl

restarted civet

moosebuild · 2022-11-21T18:16:54Z

Job Test CentOS 8 on 164e372 : invalidated by @joshua-cogliati-inl

restarted civet

moosebuild · 2022-11-21T18:17:04Z

Job Test Fedora 31 on 164e372 : invalidated by @joshua-cogliati-inl

restarted civet

moosebuild · 2022-11-21T18:17:12Z

Job Test Fedora 32 on 164e372 : invalidated by @joshua-cogliati-inl

restarted civet

moosebuild · 2022-11-21T18:17:19Z

Job Test Ubuntu 16 on 164e372 : invalidated by @joshua-cogliati-inl

restarted civet

moosebuild · 2022-11-21T18:17:25Z

Job Test Ubuntu 18-2 Python 3 on 164e372 : invalidated by @joshua-cogliati-inl

restarted civet

moosebuild · 2022-11-21T18:17:31Z

Job Test Ubuntu 20-2 Optional on 164e372 : invalidated by @joshua-cogliati-inl

restarted civet

moosebuild · 2022-11-21T20:40:42Z

Job Test Fedora 31 on 164e372 : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:40:48Z

Job Test Fedora 32 on 164e372 : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:40:55Z

Job Test Ubuntu 16 on 164e372 : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:41:04Z

Job Test Ubuntu 20-2 Optional on 164e372 : invalidated by @joshua-cogliati-inl

mystery failure

moosebuild · 2022-11-21T20:47:51Z

Job Test Ubuntu 18-2 Python 3 on 164e372 : invalidated by @joshua-cogliati-inl

mystery failure

…changes.

wangcj05

PR looks good.

wangcj05 · 2022-11-28T16:46:01Z

checklist is good, and tests are green. PR can be merged.

joshua-cogliati-inl added 2 commits September 29, 2022 09:46

Updating to ray2

963f55e

More debugging info.

743a8f0

Moving updating pythonpath to anytime ray is used.

ca0493d

joshua-cogliati-inl added 3 commits October 4, 2022 13:52

Merge remote-tracking branch 'origin/devel' into update_to_ray2

b0a96c8

Removing debug verbosity.

a008ff0

Use correct directory.

7d6c2af

Merge remote-tracking branch 'origin/devel' into update_to_ray2

6bbda33

wangcj05 added RAVENv2.2 for RAVENv2.2 Release RAVENv2.3 for RAVEN 2.3 Release and removed RAVENv2.2 for RAVENv2.2 Release labels Nov 16, 2022

Try with any ray 2.

164e372

joshua-cogliati-inl added 6 commits November 21, 2022 14:08

Merge remote-tracking branch 'origin/devel' into update_to_ray2

074272d

Try windows with 3.8

23e349a

Try python 3.9

79a950e

Adding comment.

7e3ac9a

Switch only linux and mac to ray 2.

12e9992

Switching to specifying ray 2.1 since ray sometimes has incompatible …

b5ef4c9

…changes.

wangcj05 approved these changes Nov 28, 2022

View reviewed changes

wangcj05 merged commit ea9c8a9 into idaholab:devel Nov 28, 2022

joshua-cogliati-inl deleted the update_to_ray2 branch November 28, 2022 16:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating to ray2 #1982

Updating to ray2 #1982

joshua-cogliati-inl commented Sep 29, 2022 •

edited by wangcj05

Loading

moosebuild commented Sep 30, 2022

moosebuild commented Sep 30, 2022

moosebuild commented Sep 30, 2022

moosebuild commented Sep 30, 2022

moosebuild commented Oct 3, 2022

joshua-cogliati-inl commented Oct 3, 2022

moosebuild commented Oct 3, 2022

moosebuild commented Oct 3, 2022

moosebuild commented Oct 3, 2022

moosebuild commented Oct 3, 2022

moosebuild commented Oct 4, 2022

wangcj05 commented Oct 5, 2022

moosebuild commented Oct 5, 2022

moosebuild commented Nov 15, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

wangcj05 left a comment

wangcj05 commented Nov 28, 2022

Updating to ray2 #1982

Updating to ray2 #1982

Conversation

joshua-cogliati-inl commented Sep 29, 2022 • edited by wangcj05 Loading

Pull Request Description

What issue does this change request address?

What are the significant changes in functionality due to this change request?

For Change Control Board: Change Request Review

moosebuild commented Sep 30, 2022

moosebuild commented Sep 30, 2022

moosebuild commented Sep 30, 2022

moosebuild commented Sep 30, 2022

moosebuild commented Oct 3, 2022

joshua-cogliati-inl commented Oct 3, 2022

moosebuild commented Oct 3, 2022

moosebuild commented Oct 3, 2022

moosebuild commented Oct 3, 2022

moosebuild commented Oct 3, 2022

moosebuild commented Oct 4, 2022

wangcj05 commented Oct 5, 2022

moosebuild commented Oct 5, 2022

moosebuild commented Nov 15, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

moosebuild commented Nov 21, 2022

wangcj05 left a comment

Choose a reason for hiding this comment

wangcj05 commented Nov 28, 2022

joshua-cogliati-inl commented Sep 29, 2022 •

edited by wangcj05

Loading