Skip to content

Commit 3423483

Browse files
cpagravelpull[bot]
cpagravel
authored and
pull[bot]
committed
Chef - Add sample_app_util for parsing zap files (#19087)
1 parent 76ddcfe commit 3423483

10 files changed

+8527
-1
lines changed

‎.restyled.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ exclude:
7070
- "src/controller/python/chip/clusters/Objects.py" # generated file, no point to restyle
7171
- "src/controller/python/chip/clusters/CHIPClusters.py" # generated file, no point to restyle
7272
- "scripts/idl/tests/outputs/**/*" # Matches generated output 1:1
73+
- "examples/chef/sample_app_util/test_files/*.yaml"
7374
- "examples/chef/zzz_generated/**/*"
7475

7576

‎examples/chef/BUILD.gn

+13-1
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,24 @@ import("$dir_pw_build/python.gni")
2121
pw_python_package("chef") {
2222
setup = [ "setup.py" ]
2323

24+
inputs = [
25+
"sample_app_util/test_files/sample_zap_file.zap",
26+
"sample_app_util/test_files/sample_zap_file_hashmeta.yaml",
27+
]
28+
2429
sources = [
2530
"__init__.py",
2631
"chef.py",
2732
"constants.py",
33+
"sample_app_util/__init__.py",
34+
"sample_app_util/sample_app_util.py",
35+
"sample_app_util/test_zap_file_parser.py",
36+
"sample_app_util/zap_file_parser.py",
2837
"stateful_shell.py",
2938
]
3039

31-
tests = [ "test_stateful_shell.py" ]
40+
tests = [
41+
"test_stateful_shell.py",
42+
"sample_app_util/test_zap_file_parser.py",
43+
]
3244
}

‎examples/chef/sample_app_util/README.md

+195
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
# Chef Build Conventions
2+
3+
## Overview
4+
5+
---
6+
7+
It is convenient to follow some naming and build conventions for Chef tool due
8+
to the large volume of sample apps that may be created and the ambiguity that
9+
may result from arbitrary names.
10+
11+
There are three components to the convention proposed here:
12+
13+
1. The naming convention for the sample matter device and clusters (referred to
14+
here as the `sample app`).
15+
2. The naming convention to use for the build files which will be flashed on the
16+
devices.
17+
3. The usage of metadata files that shall accompany build files to provide more
18+
detailed information about builds.
19+
20+
The convention proposed here should be adopted by the zap files provided in
21+
`examples/chef/devices` and the builds generated from Chef tool in CI.
22+
23+
## Limitations
24+
25+
---
26+
27+
The largest filename that can be used on MacOS and Linux is 255 characters. If a
28+
sample app name would exceed this limit by following this convention, then the
29+
sample app should be given an arbitrary name.
30+
31+
This limitation is called out, but, with the given naming conventions, this
32+
should rarely happen.
33+
34+
## Convention
35+
36+
---
37+
38+
### Sample App Naming Convention
39+
40+
Sample apps should be named by concatenating the name of all endpoints in the
41+
order of their index. Endpoint names are separated by underscores (`_`) and a 10
42+
character hash[^hash_note] of the sample app metadata is appended to the end.
43+
44+
Valid sample app names conform to the following format:
45+
46+
```
47+
<endpoint_0>_<endpoint_1>_<hash>
48+
```
49+
50+
For example, here are some valid names:
51+
52+
```
53+
rootnode_extendedcolorlight_H1l9gnQDYl
54+
rootnode_speaker_8qRQaEj0Hy
55+
rootnode_lightsensor_L6dEbmVDah
56+
rootnode_dimmablelight_rWsDiwzw2t
57+
rootnode_pressuresensor_03quf7tPOL
58+
rootnode_flowsensor_ixbAboycie
59+
rootnode_windowcovering_b9QoiScjOq
60+
rootnode_doorlock_d5wtU7sjFR
61+
rootnode_thermostat_KuQYArmwl7
62+
rootnode_dimmablelight_7pNE3GVarn
63+
rootnode_temperaturesensor_i0wGnDVUAc
64+
rootnode_occupancysensor_wyGeQSokNp
65+
rootnode_humiditysensor_pv0comNKyT
66+
bridgednode_temperaturesensor_onofflight_onoffpluginunit_MI9DSdkH8H
67+
```
68+
69+
[^hash_note]:
70+
71+
The 10 character hash is a base64 encoding of the md5 hash generated by
72+
digesting the JSON string encoding of the metadata information. The code for
73+
generating the hash can be found in `generate_hash` in
74+
[zap_file_parser](zap_file_parser.py) There are some notable details
75+
here: 1) The full base64 encoded hash is 16 characters, but only 10 are
76+
used. This still gives us a sufficiently low probability of collision (~1.2
77+
x 10^-8). 2) `_` and `-` are replaced in the base64 encoding because they
78+
have other uses in the naming. 3) Platform specific information is omitted
79+
from the hash. E.g. the networking_commissioning cluster is excluded. This
80+
is to make the hashes platform agnostic.
81+
82+
### Sample App Build Naming Convention
83+
84+
The sample app builds formats will be named by pre-pending the zap file name
85+
(described above) with the platform and appending connectivity info.
86+
87+
Valid build names conform to the following format:
88+
89+
```
90+
<platform>_<sample_app_name>
91+
```
92+
93+
Note that `<sample_app_name>` follows the convention:
94+
`<endpoint_0>_<endpoint_1>_<hash>`.
95+
96+
Together that is:
97+
98+
```
99+
<platform>_<endpoint_0>_<endpoint_1>_<hash>
100+
```
101+
102+
The list of platforms supported here (as of writing this) are:
103+
104+
```
105+
m5stack
106+
brd4161a
107+
nrf52840dk
108+
linux_x86
109+
```
110+
111+
For example, here are some valid names:
112+
113+
```
114+
m5stack_rootnode_humiditysensor_pv0comNKyT
115+
brd4161a_rootnode_humiditysensor_pv0comNKyT
116+
nrf52840dk_rootnode_humiditysensor_pv0comNKyT
117+
linux_x86_rootnode_humiditysensor_pv0comNKyT
118+
```
119+
120+
### Metadata file convention
121+
122+
Metadata files are `yaml` files that should accompany build files.
123+
124+
The metadata files have a structure as follows:
125+
126+
```
127+
- <endpoint_0_name>:
128+
client_clusters:
129+
<client_cluster_name>:
130+
attributes:
131+
<attribute_name>: <attribute_value>
132+
...
133+
commands:
134+
- <command_name>
135+
- ...
136+
server_clusters:
137+
<server_cluster_name>:
138+
attributes:
139+
<attribute_name>: <attribute_value>
140+
...
141+
commands:
142+
- <command_name>
143+
- ...
144+
- <endpoint_1_name>: ...
145+
```
146+
147+
For an example, see [sample_zap_file.yaml](test_files/sample_zap_file.yaml)
148+
which was generated from [sample_zap_file.zap](test_files/sample_zap_file.zap).
149+
150+
Note that it is more readable in `yaml` format. Since hashes are generated from
151+
the metadata info, additional conventions are needed to ensure consistency for
152+
the metadata structure.
153+
154+
The following conventions are used:
155+
156+
- All lists are sorted alphabetically.
157+
- If a list contains dictionaries, it will be sorted by the "name" key. If it
158+
does not contain "name" key, it will be sorted by the first key common to
159+
all dictionaries that comes first alphabetically.
160+
- The list of endpoints is excluded from the above conventions. Endpoints are
161+
ordered according to their endpoint number; here, the endpoint number is the
162+
same as the order they are read from the zap file.
163+
164+
As an example, take a look at
165+
[sample_zap_file.yaml](test_files/sample_zap_file.yaml)
166+
167+
## Utility Usage
168+
169+
---
170+
171+
There are a few primary usage cases for the utility
172+
[sample_app_util.py](sample_app_util.py). Details are provided by using
173+
`python sample_app_util.py zap --help`. Below is a summary.
174+
175+
| Command | Description |
176+
| ---------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
177+
| `python sample_app_util.py zap <zap_file> --generate-name` | Generates the name for a zap file per the specified convention |
178+
| `python sample_app_util.py zap <zap_file> --rename-file` | Renames the zap file per specified convention |
179+
| `python sample_app_util.py zap <zap_file> --generate-metadata [output_path]` | Generates the metadata file adjacent to the zap file with `.yaml` extension. If `[output_path]` is provided then the metadata file will be stored at the location specified. |
180+
181+
## Running Tests
182+
183+
---
184+
185+
Navigate to the base directory of this README.
186+
187+
```
188+
cd <project_root>/examples/chef/sample_app_util
189+
```
190+
191+
Run unit tests.
192+
193+
```
194+
python -m unittest
195+
```

‎examples/chef/sample_app_util/__init__.py

Whitespace-only changes.

‎examples/chef/sample_app_util/matter_device_types.json

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
{
2+
"Root Node": 22,
3+
"Power Source": 17,
4+
"OTA Requestor": 18,
5+
"OTA Provider": 20,
6+
"Aggregator": 14,
7+
"Bridged Node": 19,
8+
"On/Off Light": 256,
9+
"Dimmable Light": 257,
10+
"Color Temperature Light": 268,
11+
"Extended Color Light": 269,
12+
"On/Off Plug-in Unit": 266,
13+
"Dimmable Plug-In Unit": 267,
14+
"Pump": 771,
15+
"On/Off Light Switch": 259,
16+
"Dimmer Switch": 260,
17+
"Color Dimmer Switch": 261,
18+
"Control Bridge": 2112,
19+
"Pump Controller": 772,
20+
"Generic Switch": 15,
21+
"Contact Sensor": 21,
22+
"Light Sensor": 262,
23+
"Occupancy Sensor": 263,
24+
"Temperature Sensor": 770,
25+
"Pressure Sensor": 773,
26+
"Flow Sensor": 774,
27+
"Humidity Sensor": 775,
28+
"On/Off Sensor": 2128,
29+
"Door Lock": 10,
30+
"Door Lock Controller": 11,
31+
"Window Covering": 514,
32+
"Window Covering Controller": 515,
33+
"Heating/Cooling Unit": 768,
34+
"Thermostat": 769,
35+
"Fan": 43,
36+
"Casting Video Player": 35,
37+
"Speaker": 34,
38+
"Content App": 36,
39+
"Basic Video Player": 40,
40+
"Casting Video Client": 41,
41+
"Video Remote Control": 42,
42+
"Mode Select": 39
43+
}

‎examples/chef/sample_app_util/sample_app_util.py

+71
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
"""Utility to work with sample app device files.
2+
3+
This utility helps with the following:
4+
- Parsing sample app device files.
5+
- Producing metadata files from sample app device files.
6+
- Generating names for sample app device files.
7+
8+
Usage:
9+
python sample_app_util.py zap <ZAP_FILE> [options]
10+
11+
python sample_app_util.py zap test_files/sample_zap_file.zap --generate-name
12+
python sample_app_util.py zap test_files/sample_zap_file.zap --generate-metadata
13+
"""
14+
15+
import argparse
16+
import os
17+
import shutil
18+
19+
import zap_file_parser
20+
21+
22+
def zap_cmd_handler(args: argparse.Namespace) -> None:
23+
"""Handles args for zap_cmd_parser."""
24+
zap_file_path = args.zap_file
25+
if args.generate_name:
26+
print(zap_file_parser.generate_name(zap_file_path))
27+
elif args.rename_file:
28+
name = zap_file_parser.generate_name(zap_file_path)
29+
dirpath = os.path.dirname(zap_file_path)
30+
hash_string = zap_file_parser.generate_hash(zap_file_path)
31+
output_path = os.path.join(dirpath, f"{name}-{hash_string}.zap")
32+
shutil.move(zap_file_path, output_path)
33+
print(f"Renamed from: {zap_file_path} to {output_path}")
34+
elif args.generate_hash_metadata:
35+
created_file = zap_file_parser.generate_hash_metadata_file(zap_file_path)
36+
print(f"Created {created_file}")
37+
38+
39+
parser = argparse.ArgumentParser()
40+
subparsers = parser.add_subparsers(dest="command")
41+
subparsers.required = True
42+
43+
zap_cmd_parser = subparsers.add_parser("zap", help="Command to operate on zap files.")
44+
zap_cmd_parser.add_argument(
45+
"zap_file", metavar="ZAP_FILE", type=str, help="Zap file to generate name for.")
46+
47+
zap_cmd_group = zap_cmd_parser.add_mutually_exclusive_group()
48+
49+
zap_cmd_group.add_argument(
50+
"--generate-name", action="store_true",
51+
help="Print the name file name according to the name convention"
52+
)
53+
54+
zap_cmd_parser.add_argument(
55+
"--generate-hash-metadata", action="store_true",
56+
help=(
57+
"Generate the hash metadata file which provide information about what was included in "
58+
"the hash digest.")
59+
)
60+
61+
zap_cmd_group.add_argument(
62+
"--rename-file", action='store_true',
63+
help="Rename the target zap file according to name convention."
64+
)
65+
66+
zap_cmd_parser.set_defaults(func=zap_cmd_handler)
67+
68+
69+
if __name__ == "__main__":
70+
args = parser.parse_args()
71+
args.func(args)

0 commit comments

Comments
 (0)