Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Production-ready druid cluster spec #200

Merged
merged 1 commit into from
Jan 15, 2025
Merged

Add Production-ready druid cluster spec #200

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ The operator simplifies the management of Druid clusters with its custom logic t
* [Example Specs](./examples.md)
* [Developer Documentation](./dev_doc.md)
* [Migration To Kubebuilder V3 in the Upcoming Version](./kubebuilder_v3_migration.md)

---

:warning: You won't find any documentation about druid itself in this repository.
Expand All @@ -21,3 +22,20 @@ If you need details about how to architecture your druid cluster you can consult
* [Druid introduction](<https://druid.apache.org/docs/latest/design/index.html>)
* [Druid architecture](https://druid.apache.org/docs/latest/design/architecture.html)
* [Druid configuration reference](https://druid.apache.org/docs/latest/configuration/index.html)

---

[German company iunera has published their druid cluster spec](https://www.iunera.com/) in github which is used in the context of a software project by the German Ministry for Digital and Transport. The spec have the following features:

* Kubernetes-native Druid
* K8S jobs instead of Middlemanager with separated [pod-templates](https://github.com/iunera/druid-cluster-config/blob/main/kubernetes/druid/druidcluster/podTemplates/default-task-template.yaml)
* [Service Discovery by Kubernetes](https://github.com/iunera/druid-cluster-config/blob/main/kubernetes/druid/druidcluster/iuneradruid-cluster.yaml#L172) aka. no zookeeper
* [HPA for historical nodes](https://github.com/iunera/druid-cluster-config/blob/main/kubernetes/druid/druidcluster/hpa.yaml) / extended [Metrics Exporter](https://github.com/iunera/druid-cluster-config/blob/main/kubernetes/druid/metrics/druid-exporter.helm.yaml)
* Multiple [Authenticator/Authorizer](https://github.com/iunera/druid-cluster-config/blob/main/kubernetes/druid/druidcluster/iuneradruid-cluster.yaml#L88) (Basic Auth and Azure AD Authentication with pac4j)
* [Examples](https://github.com/iunera/druid-cluster-config/tree/main/_authentication-and-authorization-druid) for authorization and authentication
* Based on druid-operator and [flux-cd](https://fluxcd.io/flux/)
* Secrets managed by [SOPS](https://fluxcd.io/flux/guides/mozilla-sops/) and [ingested as Environment Variables](https://github.com/iunera/druid-cluster-config/blob/main/kubernetes/druid/druidcluster/iuneradruid-cluster.yaml#L245)
* Postgres as Metadata Store (incl. [Helmchart Config](https://github.com/iunera/druid-cluster-config/blob/main/kubernetes/druid/postgres/postgres.helm.yaml))
* All endpoints TLS encrypted incl. [Howto](https://github.com/iunera/druid-cluster-config/blob/main/README.md#cluster-internal-tls-encryption)

Link to the complete config file: https://github.com/iunera/druid-cluster-config
Loading