Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code quality improvements to the Measurement Plug-In Client generator #901

Merged
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
a2dc76c
fix: add improvements
Jotheeswaran-Nandagopal Sep 19, 2024
f75401b
fix: update pyproject.toml
Jotheeswaran-Nandagopal Sep 19, 2024
1ca6efe
Merge branch 'main' into users/jothees/client-generator-improvements
MounikaBattu17 Sep 19, 2024
b02ab87
Merge branch 'main' into users/jothees/client-generator-improvements
Jotheeswaran-Nandagopal Sep 19, 2024
3c917ec
fix: resolve comments
Jotheeswaran-Nandagopal Sep 19, 2024
d22789b
fix: update stubs
Jotheeswaran-Nandagopal Sep 20, 2024
826b662
doc: update comment string
Jotheeswaran-Nandagopal Sep 20, 2024
535d42a
refactor: string concatination
Jotheeswaran-Nandagopal Sep 20, 2024
06a4445
Merge branch 'main' into users/jothees/client-generator-improvements
Jotheeswaran-Nandagopal Sep 20, 2024
d74944f
fix: resolve merge conflict
Jotheeswaran-Nandagopal Sep 20, 2024
7f7de09
fix: update doc strings
Jotheeswaran-Nandagopal Sep 20, 2024
63deba0
remove: namespace in proto
Jotheeswaran-Nandagopal Sep 20, 2024
ed30b02
fix: move enum type conversion to mako
Jotheeswaran-Nandagopal Sep 20, 2024
b029d60
fix: automate stubs generation in generator tests
Jotheeswaran-Nandagopal Sep 20, 2024
2172c90
fix: update pyproject
Jotheeswaran-Nandagopal Sep 20, 2024
2fe7c72
fix: update stubs
Jotheeswaran-Nandagopal Sep 20, 2024
2b3a8ea
fix: update mypy override
Jotheeswaran-Nandagopal Sep 20, 2024
a811110
fix: update proto
Jotheeswaran-Nandagopal Sep 20, 2024
4dd2b62
fix: update exception message
Jotheeswaran-Nandagopal Sep 20, 2024
926f723
fix: update mako
Jotheeswaran-Nandagopal Sep 20, 2024
7247618
update: formatting
Jotheeswaran-Nandagopal Sep 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/check_nimg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,13 @@ jobs:
run: poetry run mypy --platform win32
- name: Bandit security checks (ni-measurement-plugin-sdk-generator, example_renders)
run: poetry run bandit -c pyproject.toml -r ni_measurement_plugin_sdk_generator tests/test_assets/example_renders
- name: Generate gRPC stubs
run: |
find tests/utilities/measurements/non_streaming_data_measurement/_stubs -name \*_pb2.py\* -o -name \*_pb2_grpc.py\* -delete
poetry run python scripts/generate_grpc_stubs.py
- name: Check for out-of-date gRPC stubs
run: git diff --exit-code
- name: Revert gRPC stubs
run: |
git clean -dfx tests/utilities/measurements/non_streaming_data_measurement/_stubs
git restore tests/utilities/measurements/non_streaming_data_measurement/_stubs
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ The generated client includes four APIs: `measure`, `stream_measure`, `register_

1. Make sure the required measurement service is running before interacting with it via the client.

2. Use the client APIs from the ["Developing a Minimal Python MeasurementClient"](#developing-a-minimal-python-measurement-client) section.
2. Use the client APIs from the ["Generating a Minimal Python Measurement Client"](#generating-a-minimal-python-measurement-client) section.

1. For non-streaming measurements, use the `measure` method.

Expand Down
2 changes: 1 addition & 1 deletion packages/generator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ The generated client includes four APIs: `measure`, `stream_measure`, `register_

1. Make sure the required measurement service is running before interacting with it via the client.

2. Use the client APIs from the ["Developing a Minimal Python MeasurementClient"](#developing-a-minimal-python-measurement-client) section.
2. Use the client APIs from the ["Generating a Minimal Python Measurement Client"](#generating-a-minimal-python-measurement-client) section.

1. For non-streaming measurements, use the `measure` method.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
"""Utilizes command line args to create a Measurement Plug-In Client using template files."""

import pathlib
import re
from enum import Enum
from typing import Any, Dict, List, Optional, Type

Expand Down Expand Up @@ -39,18 +38,12 @@ def _render_template(template_name: str, **template_args: Any) -> bytes:
return template.render(**template_args)


def _replace_enum_class_type(output: str) -> str:
pattern = "<enum '([^']+)'>"
return re.sub(pattern, r"\1", output)


def _create_file(
template_name: str, file_name: str, directory_out: pathlib.Path, **template_args: Any
) -> None:
output_file = directory_out / file_name

output = _render_template(template_name, **template_args).decode("utf-8")
output = _replace_enum_class_type(output)
formatted_output = black.format_str(
src_contents=output,
mode=black.Mode(line_length=100),
Expand Down Expand Up @@ -231,7 +224,7 @@ def _create_clients(
@optgroup.option(
"-s",
"--measurement-service-class",
help="Creates Python Measurement Plug-In Client for the given measurement services.",
help="Creates Python Measurement Plug-In Client for the given measurement service classes.",
multiple=True,
)
@optgroup.option(
Expand Down Expand Up @@ -276,9 +269,6 @@ def create_client(
"""Generates a Python Measurement Plug-In Client module for the measurement service.

You can use the generated module to interact with the corresponding measurement service.

MEASUREMENT_SERVICE_CLASS: Accepts one or more measurement service classes.
Provide each service class separately.
"""
if all:
_create_all_clients(directory_out)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import json
import keyword
import os
import pathlib
import re
import sys
Expand Down Expand Up @@ -154,7 +153,7 @@ def get_configuration_metadata_by_index(
deserialized_parameters = deserialize_parameters(
configuration_metadata,
metadata.measurement_signature.configuration_defaults.value,
service_class + ".Configurations",
f"{service_class}.Configurations",
)

for k, v in deserialized_parameters.items():
Expand Down Expand Up @@ -248,10 +247,7 @@ def get_configuration_parameters_with_type_and_default_values(

configuration_parameters.append(f"{parameter_name}: {parameter_type} = {default_value}")

# Use line separator and spaces to align the parameters appropriately in the generated file.
configuration_parameters_with_type_and_value = f",{os.linesep} ".join(
configuration_parameters
)
configuration_parameters_with_type_and_value = f", ".join(configuration_parameters)
parameter_names_as_str = ", ".join(parameter_names)

return (configuration_parameters_with_type_and_value, parameter_names_as_str)
Expand All @@ -262,9 +258,9 @@ def get_output_parameters_with_type(
built_in_import_modules: List[str],
custom_import_modules: List[str],
enum_values_by_type: Dict[Type[Enum], Dict[str, int]] = {},
) -> str:
) -> List[str]:
"""Returns the output parameters of the measurement with type."""
output_parameters_with_type = []
output_parameters_with_type: List[str] = []
for metadata in output_metadata.values():
parameter_name = _get_python_identifier(metadata.display_name)
parameter_type = _get_python_type_as_str(metadata.type, metadata.repeated)
Expand All @@ -289,7 +285,7 @@ def get_output_parameters_with_type(

output_parameters_with_type.append(f"{parameter_name}: {parameter_type}")

return f"{os.linesep} ".join(output_parameters_with_type)
return output_parameters_with_type


def to_ordered_set(values: Iterable[_T]) -> AbstractSet[_T]:
Expand Down Expand Up @@ -438,4 +434,4 @@ def _get_enum_class_name(name: str) -> str:
name = "".join(s.capitalize() for s in split_string)
else:
name = name[0].upper() + name[1:]
return name + "Enum"
return f"{name}Enum"
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
<%page args="class_name, display_name, configuration_metadata, output_metadata, service_class, configuration_parameters_with_type_and_default_values, measure_api_parameters, output_parameters_with_type, built_in_import_modules, custom_import_modules, enum_by_class_name, configuration_parameters_type_url"/>\
\
<%!
import re
%>

<%page args="class_name, display_name, configuration_metadata, output_metadata, service_class, configuration_parameters_with_type_and_default_values, measure_api_parameters, output_parameters_with_type, built_in_import_modules, custom_import_modules, enum_by_class_name, configuration_parameters_type_url"/>

<%
def _replace_enum_class_type(input_string: str) -> str:
"""Replace enum class type representation with the enum name."""
pattern = r"<enum '([^']+)'>"
return re.sub(pattern, r"\1", input_string)

configuration_metadata = _replace_enum_class_type(str(configuration_metadata))
if output_metadata:
output_metadata = _replace_enum_class_type(str(output_metadata))
%>

"""Generated client API for the ${display_name | repr} measurement plug-in."""

import logging
Expand Down Expand Up @@ -57,7 +72,9 @@ class ${enum_name.__name__}(Enum):
class Outputs(NamedTuple):
"""Outputs for the ${display_name | repr} measurement plug-in."""

${output_parameters_with_type}
% for output_parameter in output_parameters_with_type:
${output_parameter}
% endfor
<% output_type = "Outputs" %>\
% endif

Expand Down Expand Up @@ -102,7 +119,7 @@ class ${class_name}:

@property
def pin_map_context(self) -> PinMapContext:
"""Get the pin map context for the measurement."""
"""The pin map context for the measurement."""
return self._pin_map_context

@pin_map_context.setter
Expand Down
Loading