Skip to content

Commit 758c80d

Browse files
shireen-beanshireen-bean
authored
Merge pull request #252 from spotify/release-21.12.0
Release 21.12.0: Merge release-21.12.0 into develop
2 parents cb566a3 + dd1d856 commit 758c80d

File tree

15 files changed

+113
-23
lines changed

15 files changed

+113
-23
lines changed

‎cli/setup.cfg

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[bumpversion]
2-
current_version = 21.9.0
2+
current_version = 21.12.0
33
tag_name = cli-{new_version}
44
commit = True
55
tag = True
66
message = [cli] Bump version: {current_version} → {new_version}
77
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)
8-
serialize =
8+
serialize =
99
{major}.{minor}.{patch}
1010

1111
[bumpversion:file:src/klio_cli/__init__.py]

‎cli/src/klio_cli/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
#
1515

1616
__author__ = "The klio developers"
17-
__version__ = "21.9.0"
17+
__version__ = "21.12.0"
1818
__email__ = "opensource+klio@spotify.com"
1919
__description__ = "Main entrypoint for Klio jobs"
2020
__uri__ = "https://github.com/spotify/klio"

‎cli/src/klio_cli/utils/cli_utils.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ def error_stackdriver_logger_metrics(klio_config, direct_runner):
115115
if stackdriver_conf not in (False, None):
116116
msg = (
117117
"The Stackdriver log-based metric client has been deprecated "
118-
"since 21.3.0 and removed in 21.10.0, in favor of the Native "
118+
"since 21.3.0 and removed in 21.12.0, in favor of the Native "
119119
"metrics client.\n"
120120
"See docs on how to turn on the native client for Dataflow: "
121121
"https://docs.klio.io/en/stable/userguide/pipeline/metrics.html"

‎core/setup.cfg

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[bumpversion]
2-
current_version = 21.9.0
2+
current_version = 21.12.0
33
commit = True
44
tag = True
55
tag_name = core-{new_version}
66
message = [core] Bump version: {current_version} → {new_version}
77
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)
8-
serialize =
8+
serialize =
99
{major}.{minor}.{patch}
1010

1111
[bumpversion:file:src/klio_core/__init__.py]

‎core/src/klio_core/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
#
1515

1616
__author__ = "The klio developers"
17-
__version__ = "21.9.0"
17+
__version__ = "21.12.0"
1818
__email__ = "opensource+klio@spotify.com"
1919
__description__ = "Core klio library for common functionality"
2020
__uri__ = "https://github.com/spotify/klio"

‎docs/src/reference/cli/changelog.rst

+5-4
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
CLI Changelog
22
=============
33

4-
.. _cli-21.10.0:
4+
.. _cli-21.12.0:
55

6-
21.10.0 (UNRELEASED)
6+
21.12.0 (2021-12-14)
77
--------------------
88

9-
.. start-21.10.0
9+
.. start-21.12.0
1010
1111
Added
1212
*****
@@ -23,8 +23,9 @@ Changed
2323
*******
2424

2525
* When running a job, effective config is no longer written to ``klio-job-run-effective.yaml``, but instead to a temp file. This file no longer needs to be included in ``setup.py`` projects (See `PR 233 <https://github.com/spotify/klio/pull/233>`_).
26+
* Error out when a user tries to run a Dataflow-based job with Stackdriver log-based metrics client configured.
2627

27-
.. end-21.10.0
28+
.. end-21.12.0
2829
2930
.. _cli-21.9.0:
3031

‎docs/src/reference/core/changelog.rst

+11
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,17 @@
11
Changelog
22
=========
33

4+
.. _core-21.12.0:
5+
6+
21.12.0 (2021-12-14)
7+
--------------------
8+
9+
.. start-21.12.0
10+
11+
No changes - bump version to sync with ``21.12.0`` release.
12+
13+
.. end-21.12.0
14+
415
.. _core-21.9.0:
516

617
21.9.0 (2021-10-12)

‎docs/src/reference/executor/changelog.rst

+11
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,17 @@
11
Changelog
22
=========
33

4+
.. _exec-21.12.0:
5+
6+
21.12.0 (2021-12-14)
7+
--------------------
8+
9+
.. start-21.12.0
10+
11+
No changes - bump version to sync with ``21.12.0`` release.
12+
13+
.. end-21.12.0
14+
415
.. _exec-21.9.0:
516

617
21.9.0 (2021-10-12)

‎docs/src/reference/lib/changelog.rst

+17-6
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,30 @@
11
Changelog
22
=========
33

4-
.. _lib-21.10.0:
4+
.. _lib-21.12.0:
55

6-
21.10.0 (UNRELEASED)
6+
21.12.0 (2021-12-14)
77
--------------------
88

9-
.. start-21.10.0
9+
.. start-21.12.0
1010
11-
Changed
11+
Fixed
12+
*****
13+
14+
* Fixed bug in metrics parsing when metrics configuration was set to a bool instead of a dict.
15+
16+
Removed
17+
*******
18+
19+
* Removed deprecated Stackdriver log-based metrics client.
20+
21+
Changes
1222
*******
1323

14-
* ``KlioConfig`` is now loaded on workers from pickled main session instead of a bundled config file (See `PR 233 <https://github.com/spotify/klio/pull/233>`_).
24+
* Updated documentation related to the native metrics client.
25+
* ``KlioConfig`` is now loaded on workers from pickled main session instead of a bundled config file (See `PR 220 <https://github.com/spotify/klio/pull/220>`_).
1526

16-
.. end-21.10.0
27+
.. end-21.12.0
1728
1829
1930
.. _lib-21.9.0:

‎docs/src/release_notes/21.12.0.rst

+55
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
21.12.0
2+
=======
3+
4+
Load Klio Config from Main Session
5+
----------------------------------
6+
Klio now uses Apache Beam's main session to load the Klio config file onto workers. Klio config fields are defined in the ``klio-job.yaml`` and can also be overridden [#f1]_ when running a pipeline. The final config fields are serialized as the ``KlioConfig`` object, saved into the main session, and loaded by Dataflow workers upon starting up. As a result, Klio jobs no longer need to include ``.effective-klio-job.yaml`` and ``klio-job-run-effective.yaml`` in the job's docker image.
7+
8+
This change is dependent on a bug fix available in ``apache-beam>=2.31.0``. Klio jobs using a version lower than this should upgrade ``apache-beam``.
9+
10+
11+
Metrics
12+
-------
13+
The Stackdriver log-based metric client, which has been deprecated since March 2021, has been removed in this release.
14+
15+
16+
Verifying GCP Roles
17+
-------------------
18+
The release of ``21.12.0`` also includes minor improvements to the user experience when verifying a Klio job and adding GCP roles to a job's service account.
19+
20+
21+
Changes
22+
-------
23+
24+
:ref:`klio-cli <cli-21.12.0>`
25+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
26+
27+
.. include:: /reference/cli/changelog.rst
28+
:start-after: start-21.12.0
29+
:end-before: end-21.12.0
30+
31+
:ref:`klio <lib-21.12.0>`
32+
^^^^^^^^^^^^^^^^^^^^^^^^^
33+
34+
.. include:: /reference/lib/changelog.rst
35+
:start-after: start-21.12.0
36+
:end-before: end-21.12.0
37+
38+
39+
:ref:`klio-core <core-21.12.0>`
40+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
41+
42+
.. include:: /reference/core/changelog.rst
43+
:start-after: start-21.12.0
44+
:end-before: end-21.12.0
45+
46+
:ref:`klio-exec <exec-21.12.0>`
47+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
48+
49+
.. include:: /reference/executor/changelog.rst
50+
:start-after: start-21.12.0
51+
:end-before: end-21.12.0
52+
53+
.. rubric:: Footnotes
54+
55+
.. [#f1] https://docs.klio.io/en/stable/userguide/config/index.html?highlight=override#overriding-values

‎docs/src/release_notes/index.rst

+1
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ Release Notes
1010
.. toctree::
1111
:maxdepth: 2
1212

13+
21.12.0
1314
21.9.0
1415
21.8.0
1516
21.2.0

‎exec/setup.cfg

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[bumpversion]
2-
current_version = 21.9.0
2+
current_version = 21.12.0
33
commit = True
44
tag = True
55
tag_name = exec-{new_version}
66
message = [exec] Bump version: {current_version} → {new_version}
77
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)
8-
serialize =
8+
serialize =
99
{major}.{minor}.{patch}
1010

1111
[bumpversion:file:src/klio_exec/__init__.py]

‎exec/src/klio_exec/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
#
1515

1616
__author__ = "The klio developers"
17-
__version__ = "21.9.0"
17+
__version__ = "21.12.0"
1818
__email__ = "opensource+klio@spotify.com"
1919
__description__ = "Klio pipeline executor within a job's Docker image."
2020
__uri__ = "https://github.com/spotify/klio"

‎lib/setup.cfg

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
[bumpversion]
2-
current_version = 21.9.0
2+
current_version = 21.12.0
33
commit = True
44
tag = True
55
tag_name = lib-{new_version}
66
message = [lib] Bump version: {current_version} → {new_version}
77
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)
8-
serialize =
8+
serialize =
99
{major}.{minor}.{patch}
1010

1111
[bumpversion:file:src/klio/__init__.py]

‎lib/src/klio/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
#
1515

1616
__author__ = "The klio developers"
17-
__version__ = "21.9.0"
17+
__version__ = "21.12.0"
1818
__email__ = "opensource+klio@spotify.com"
1919
__description__ = "Conventions for Python + Apache Beam "
2020
__uri__ = "https://github.com/spotify/klio"

0 commit comments

Comments
 (0)