Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: CLI options can be supplied using env vars #18

Merged
merged 2 commits into from
Mar 6, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -16,3 +16,6 @@ python:
path: .
extra_requirements:
- doc

sphinx:
configuration: doc/conf.py
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -198,6 +198,26 @@ This is a fast, convenient, and **recommended** way to submit a job without havi
script and since everything is tracked by GridTK, you still benefit from the same
reproducibility guarantees.

### Environment Variables for CLI Options

While sbatch already allows providing values for some options through `SBATCH_` prefixed environment variables, not all options are supported. GridTK adds support for all CLI options through the `GRIDTK_SUBMIT_` prefix. For example:

```bash
# Set default email notification settings
export GRIDTK_SUBMIT_MAIL_USER=your.email@example.com
export GRIDTK_SUBMIT_MAIL_TYPE=END

# Use debug partition by default
export GRIDTK_SUBMIT_PARTITION=debug

# Now submit a job - it will use these settings automatically
gridtk submit job.sh
```

This is useful when you have a set of options that you always want to use, but don't want to specify them every time you submit a job.

Of course, other gridtk commands such as `gridtk resubmit` options can also be set using environment variables like: `export GRIDTK_RESUBMIT_STATE=ALL`.

### Job Dependencies

To submit a job that depends on another job, use the `--dependency` flag:
1 change: 1 addition & 0 deletions src/gridtk/cli.py
Original file line number Diff line number Diff line change
@@ -142,6 +142,7 @@ def _job_filters_decorator(function):
context_settings={
"show_default": True,
"help_option_names": ["--help", "-h"],
"auto_envvar_prefix": "GRIDTK",
},
)
@click.option(
39 changes: 39 additions & 0 deletions tests/test_gridtk.py
Original file line number Diff line number Diff line change
@@ -202,6 +202,45 @@ def test_submit_wrap(mock_check_output, runner):
)


@patch("subprocess.check_output")
def test_submit_with_env_vars(mock_check_output: Mock, runner):
"""Test that environment variables with GRIDTK_SUBMIT_ prefix are properly converted to CLI options."""
mock_check_output.return_value = _sbatch_output(123456789)
with runner.isolated_filesystem():
result = runner.invoke(
cli,
["submit", "job.sh"],
env={
"GRIDTK_SUBMIT_MAIL_USER": "test@example.com",
"GRIDTK_SUBMIT_MAIL_TYPE": "END",
"GRIDTK_SUBMIT_PARTITION": "debug",
},
)

assert_click_runner_result(result)
assert "1" in result.output

mock_check_output.assert_called_with(
[
"sbatch",
"--job-name",
"gridtk",
"--output",
"logs/gridtk.%j.out",
"--error",
"logs/gridtk.%j.out",
"--mail-type",
"END",
"--mail-user",
"test@example.com",
"--partition",
"debug",
"job.sh",
],
text=True,
)


@patch("subprocess.check_output")
def test_submit_triple_dash(mock_check_output: Mock, runner):
mock_check_output.return_value = _sbatch_output(123456789)