Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add metrics to proxy #1017

Merged
merged 39 commits into from
Jan 3, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
06be9fb
feat: add metrics to proxy
SantiagoPittella Dec 12, 2024
ff098ac
review: update RequestContext documentation
SantiagoPittella Dec 19, 2024
5f6096e
review: replace lazy_static with LazyLock
SantiagoPittella Dec 20, 2024
c8506a4
review: use always localhost for metrics host
SantiagoPittella Dec 23, 2024
9c07da7
Merge branch 'next' into santiagopittella-add-metrics-to-proxy
bobbinth Dec 27, 2024
00a76b2
review: improve separators in metrics definition
SantiagoPittella Dec 27, 2024
e486ffd
review: re-add metrics host configuration and default to localhost
SantiagoPittella Dec 27, 2024
ae669d2
review: update prometheus.yml
SantiagoPittella Dec 27, 2024
9c2836e
review: add information about grafana dashboard creation and export t…
SantiagoPittella Dec 27, 2024
01ddba2
review: add documentation on local installation of prometheus and gra…
SantiagoPittella Dec 27, 2024
dd8a928
review: add units to histograms
SantiagoPittella Dec 30, 2024
1729d6c
review: rename the tag in prometheus.yml to tx_prover
SantiagoPittella Dec 30, 2024
308612c
review: fix WORKER_COUNT metric update
SantiagoPittella Dec 30, 2024
5e12d97
review: fix WORKER_UTILIZATION updates
SantiagoPittella Dec 30, 2024
89c6933
review: update RequestQueue docs
SantiagoPittella Dec 30, 2024
ac33f91
review: move WORKER_UNHEALTHY update logic
SantiagoPittella Dec 30, 2024
021923c
review: fix WORKER_UTILIZATION desc
SantiagoPittella Dec 30, 2024
56b2621
review: rename WORKER_UTILIZATION to WORKER_BUSY
SantiagoPittella Dec 30, 2024
64bb910
review: add metrics host to readme
SantiagoPittella Dec 30, 2024
0ab9b38
review: mention config file by name
SantiagoPittella Dec 30, 2024
edb6092
review: update doc about prometheus and grafana
SantiagoPittella Dec 30, 2024
80a02e7
review: add missing config field to readme example
SantiagoPittella Dec 30, 2024
411b023
docs: improve prometheus.yml docs
SantiagoPittella Dec 30, 2024
6d9c2f5
chore: add safety comment to metrics
SantiagoPittella Dec 30, 2024
2b9fc91
chore: initialize metrics on load balancer instance
SantiagoPittella Dec 30, 2024
749d5f5
chore: update dashboard
SantiagoPittella Dec 30, 2024
44fe4f0
review: change unhealthy workers metric description
SantiagoPittella Dec 30, 2024
370eb99
review: add missing field to README example
SantiagoPittella Jan 2, 2025
8066e46
review: rewrite Grafana introduction on readme
SantiagoPittella Jan 2, 2025
429b28d
review: change WORKER_COUNT metric description
SantiagoPittella Jan 2, 2025
82dbe5e
review: do not count update worker requests for metrics
SantiagoPittella Jan 2, 2025
7e4a3c5
review: move busy workers metric update
SantiagoPittella Jan 2, 2025
bc577dc
review: update grafana dashboard
SantiagoPittella Jan 2, 2025
ce0892d
review: update dashboard
SantiagoPittella Jan 2, 2025
07c999e
review: move metric update to create response function
SantiagoPittella Jan 3, 2025
5295fd2
chore: address lint errors
SantiagoPittella Jan 3, 2025
3e35509
Merge branch 'next' into santiagopittella-add-metrics-to-proxy
SantiagoPittella Jan 3, 2025
61b7074
review: rename grafana dashboard file
SantiagoPittella Jan 3, 2025
d9c1d40
review: update dashboard
SantiagoPittella Jan 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

### Changes

- Added tracing to the `miden-tx-prover` CLI (#1014).
- Added health check endpoints to the prover service (#1006).
- Implemented serialization for `AccountHeader` (#996).
- Updated Pingora crates to 0.4 and added polling time to the configuration file (#997).
Expand All @@ -19,6 +18,8 @@
- [BREAKING] Added `miden::note::get_script_hash` procedure (#995).
- [BREAKING] Refactor error messages in `miden-lib` and `miden-tx` and use `thiserror` 2.0 (#1005).
- Removed workers list from the proxy configuration file (#1018).
- Added tracing to the `miden-tx-prover` CLI (#1014).
- Added metrics to the `miden-tx-prover` proxy (#1017).

## 0.6.2 (2024-11-20)

Expand Down
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions bin/tx-prover/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ figment = { version = "0.10", features = ["toml", "env"] }
miden-lib = { workspace = true, default-features = false }
miden-objects = { workspace = true, default-features = false }
miden-tx = { workspace = true, default-features = false }
lazy_static = "1.5"
prometheus = "0.13"
prost = { version = "0.13", default-features = false, features = ["derive"] }
reqwest = { version = "0.11" }
serde = { version = "1.0", features = ["derive"] }
Expand Down
20 changes: 20 additions & 0 deletions bin/tx-prover/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,26 @@ Then access the Jaeger UI at `http://localhost:16686/`.

If Docker is not an option, Jaeger can also be set up directly on your machine or hosted in the cloud. See the [Jaeger documentation](https://www.jaegertracing.io/docs/) for alternative installation methods.

## Metrics

The proxy includes a service that provides metrics for [Prometheus](https://prometheus.io/docs/introduction/overview/). This service is always enabled and uses the host and port defined in the configuration file. The metrics are available at the `/metrics` endpoint.

To consume and display the metrics, you can use Prometheus and Grafana. The simplest way to install Prometheus and Grafana is by using Docker containers. To do so, run:

```bash
docker run \
-d \
-p 9090:9090 \
-v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml \
prom/prometheus

docker run -d -p 3000:3000 --name grafana grafana/grafana-enterprise:latest
```

A prometheus configuration file is provided in this repository, you will need to modify the `scrape_configs` section to include the host and port of the proxy service.

Then, to add the new Prometheus collector as a datasource for Grafana, you can [follow this tutorial](https://grafana.com/docs/grafana-cloud/connect-externally-hosted/existing-datasource/). A Grafana dashboard under the name `proxy_grafana_dashboard.json` is provided, see this [link](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/import-dashboards/) to import it.

## Features

Description of this crate's feature:
Expand Down
11 changes: 11 additions & 0 deletions bin/tx-prover/prometheus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).

# A scrape configuration containing exactly one endpoint to scrape:
scrape_configs:
- job_name: "proxy"
# Here you need to specify the address of the Prometheus service endpoint in the proxy
static_configs:
- targets: ["127.0.0.1:6192"]
Loading
Loading