Skip to content

Commit 452bf58

Browse files
robertbartelaaraney
authored andcommitted
Optimize FromFilesInitialDataAdder.
Update so that call to manager.add_data passes a buffered/binary reader object instead of just reading all the bytes up front and passing those.
1 parent ffd2132 commit 452bf58

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

python/services/dataservice/dmod/dataservice/initial_data_adder_impl.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ def _add(path: Path, dest: Optional[str] = None, add_root: Optional[Path] = None
6868
elif not dest:
6969
dest = path.relative_to(add_root)
7070

71-
if not self._dataset_manager.add_data(dataset_name=self._dataset_name, dest=dest, data=path.read_bytes(),
71+
if not self._dataset_manager.add_data(dataset_name=self._dataset_name, dest=dest, data=io.open(path, "rb"),
7272
domain=original_domain):
7373
msg = f"{self.__class__.__name__} failed to add item {dest} under {str(self._source_path)}"
7474
raise DmodRuntimeError(msg)

0 commit comments

Comments
 (0)