Merge branch 'main' into feat/singleton-connection

Signed-off-by: lorenabalan <lorena.balan@quantumblack.com>

Author: Lorena Bălan

.github/dco.yml

@@ -0,0 +1,2 @@
+require:
+  members: false

Makefile

@@ -55,3 +55,9 @@ uninstall-pre-commit:
 
 print-python-env:
 	@./tools/print_env.sh
+
+sign-off:
+	echo "git interpret-trailers --if-exists doNothing \c" >> .git/hooks/commit-msg
+	echo '--trailer "Signed-off-by: $$(git config user.name) <$$(git config user.email)>" \c' >> .git/hooks/commit-msg
+	echo '--in-place "$$1"' >> .git/hooks/commit-msg
+	chmod +x .git/hooks/commit-msg

docs/source/05_data/01_data_catalog.md

@@ -313,6 +313,27 @@ dev_abs:
   account_name: accountname
   account_key: key
 ```
+Example 16: Loading a CSV file stored in a remote location through SSH
+
+```eval_rst
+.. note::  This example requires [Paramiko](https://www.paramiko.org) to be installed (`pip install paramiko`).
+```
+```yaml
+cool_dataset:
+  type: pandas.CSVDataSet
+  filepath: "sftp:///path/to/remote_cluster/cool_data.csv"
+  credentials: cluster_credentials
+```
+All parameters required to establish the SFTP connection can be defined through `fs_args` or in `credentials.yml` as follows:
+
+```yaml
+cluster_credentials:
+  username: my_username
+  host: host_address
+  port: 22
+  password: password
+```
+The list of all available parameters is given in the [Paramiko documentation](https://docs.paramiko.org/en/2.4/api/client.html#paramiko.client.SSHClient.connect).
 
 ## Creating a Data Catalog YAML configuration file via CLI
 

docs/source/08_logging/01_logging.md

@@ -18,6 +18,10 @@ log.warning("Issue warning")
 log.info("Send information")
 ```
 
+```eval_rst
+.. note:: The name of a logger corresponds to a key in the ``loggers``  section in ``logging.yml`` (e.g. ``kedro.io``). See `Python's logging documentation <https://docs.python.org/3/library/logging.html#logger-objects>`_ for more information.
+```
+
 ## Logging for `anyconfig`
 
 By default, [anyconfig](https://github.com/ssato/python-anyconfig) library that is used by `kedro` to read configuration files emits a log message with `INFO` level on every read. To reduce the amount of logs being sent for CLI calls, default project logging configuration in `conf/base/logging.yml` sets the level for `anyconfig` logger to `WARNING`.

docs/source/14_contribution/02_developer_contributor_guidelines.md

@@ -180,7 +180,7 @@ The sign-off can be added automatically to your commit message using the `-s` op
 git commit -s -m "This is my commit message"
 ```
 
-To avoid needing to remember the `-s` flag on every commit, you might like to set up an [alias](https://git-scm.com/book/en/v2/Git-Basics-Git-Aliases) for `git commit -s`.
+To avoid needing to remember the `-s` flag on every commit, you might like to set up an [alias](https://git-scm.com/book/en/v2/Git-Basics-Git-Aliases) for `git commit -s`. Alternatively, run `make sign-off` to setup a [`commit-msg` Git hook](https://git-scm.com/docs/githooks#_commit_msg) that automatically signs off all commits (including merge commits) you make while working on the Kedro repository.
 
 If your PR is blocked due to unsigned commits then you will need to follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.
 

kedro/extras/datasets/json/json_dataset.py

@@ -21,7 +21,28 @@ class JSONDataSet(AbstractVersionedDataSet):
     """``JSONDataSet`` loads/saves data from/to a JSON file using an underlying
     filesystem (e.g.: local, S3, GCS). It uses native json to handle the JSON file.
 
-    Example:
+    Example adding a catalog entry with
+    `YAML API <https://kedro.readthedocs.io/en/stable/05_data/\
+        01_data_catalog.html#using-the-data-catalog-with-the-yaml-api>`_:
+
+    .. code-block:: yaml
+
+        >>> json_dataset:
+        >>>   type: json.JSONDataSet
+        >>>   filepath: data/01_raw/location.json
+        >>>   load_args:
+        >>>     lines: True
+        >>>
+        >>> cars:
+        >>>   type: json.JSONDataSet
+        >>>   filepath: gcs://your_bucket/cars.json
+        >>>   fs_args:
+        >>>     project: my-project
+        >>>   credentials: my_gcp_credentials
+        >>>   load_args:
+        >>>     lines: True
+
+    Example using Python API:
     ::
 
         >>> from kedro.extras.datasets.json import JSONDataSet

kedro/runner/parallel_runner.py

@@ -187,7 +187,7 @@ def _validate_nodes(cls, nodes: Iterable[Node]):
                 f"In order to utilize multiprocessing you need to make sure all nodes "
                 f"are serializable, i.e. nodes should not include lambda "
                 f"functions, nested functions, closures, etc.\nIf you "
-                f"are using custom decorators ensure they are correctly using "
+                f"are using custom decorators ensure they are correctly decorated using "
                 f"functools.wraps()."
             )
 
@@ -217,7 +217,7 @@ def _validate_catalog(cls, catalog: DataCatalog, pipeline: Pipeline):
                 f"need to make sure all data sets are serializable, i.e. data sets "
                 f"should not make use of lambda functions, nested functions, closures "
                 f"etc.\nIf you are using custom decorators ensure they are correctly "
-                f"using functools.wraps()."
+                f"decorated using functools.wraps()."
             )
 
         memory_data_sets = []