enhancement(vector): WIP — estimated JSON-encoded event size v2

[JeanMertz]

🚧

[netlify]

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	7d6ebf0
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/630909be00fbed000969d8f7

[github-actions]

Soak Test Results

Baseline: deea758
Comparison: 2d14be6
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

Changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

experiment	Δ mean	Δ mean %	confidence
syslog_log2metric_humio_metrics	-1.18MiB	-9.41	100.00%
splunk_hec_route_s3	-2.69MiB	-14.7	100.00%
syslog_loki	-3.35MiB	-23.53	100.00%
syslog_regex_logs2metric_ddmetrics	-3.08MiB	-25.3	100.00%
http_pipelines_blackhole_acks	-309.88KiB	-25.6	100.00%
syslog_humio_logs	-4.32MiB	-29.35	100.00%
syslog_splunk_hec_logs	-4.7MiB	-29.42	100.00%
datadog_agent_remap_datadog_logs	-19.85MiB	-31.81	100.00%
syslog_log2metric_splunk_hec_metrics	-5.68MiB	-33.11	100.00%
datadog_agent_remap_datadog_logs_acks	-20.33MiB	-33.26	100.00%
http_pipelines_blackhole	-574.76KiB	-33.37	100.00%
http_pipelines_no_grok_blackhole	-5.34MiB	-48.96	100.00%
datadog_agent_remap_blackhole_acks	-34.44MiB	-55.85	100.00%
datadog_agent_remap_blackhole	-36.02MiB	-56.83	100.00%

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
splunk_hec_to_splunk_hec_logs_noack	19.35KiB	0.08	89.22%	23.82MiB	469.35KiB	9.58KiB	0	0.0192373	23.84MiB	355.89KiB	7.27KiB	0	0.0145751	False	False
splunk_hec_indexer_ack_blackhole	11.93KiB	0.05	39.00%	23.75MiB	858.51KiB	17.46KiB	0	0.0352866	23.77MiB	765.15KiB	15.57KiB	0	0.031434	False	False
splunk_hec_to_splunk_hec_logs_acks	-766.69B	-0	2.38%	23.76MiB	836.35KiB	17.02KiB	0	0.0343721	23.76MiB	909.63KiB	18.5KiB	0	0.0373846	False	False
enterprise_http_to_http	653.55B	0	6.61%	23.85MiB	251.29KiB	5.13KiB	0	0.0102886	23.85MiB	280.2KiB	5.73KiB	0	0.0114719	False	False
file_to_blackhole	-85.84KiB	-0.09	70.54%	95.37MiB	2.6MiB	53.82KiB	0	0.0272167	95.29MiB	2.97MiB	61.72KiB	0	0.031152	False	False
http_to_http_noack	-30.93KiB	-0.13	88.66%	23.83MiB	510.93KiB	10.44KiB	0	0.020935	23.8MiB	809.67KiB	16.51KiB	0	0.0332178	False	False
http_to_http_json	-52.97KiB	-0.22	99.97%	23.85MiB	335.57KiB	6.85KiB	0	0.0137393	23.79MiB	630.63KiB	12.86KiB	0	0.0258762	False	False
fluent_elasticsearch	-1.55MiB	-1.95	100.00%	79.47MiB	54.38KiB	1.1KiB	0	0.000668113	77.92MiB	3.77MiB	77.42KiB	0	0.0483678	False	False
socket_to_socket_blackhole	-875.31KiB	-3.68	100.00%	23.25MiB	130.57KiB	2.67KiB	0	0.00548346	22.39MiB	143.99KiB	2.94KiB	0	0.0062777	False	False
http_text_to_http_json	-3.38MiB	-8.31	100.00%	40.72MiB	813.68KiB	16.61KiB	0	0.0195084	37.34MiB	874.42KiB	17.85KiB	0	0.0228639	False	False
syslog_log2metric_humio_metrics	-1.18MiB	-9.41	100.00%	12.5MiB	627.12KiB	12.8KiB	0	0.0489961	11.32MiB	600.44KiB	12.23KiB	0	0.0517837	False	False
http_to_http_acks	-1.93MiB	-11.24	100.00%	17.13MiB	7.69MiB	160.73KiB	0	0.448872	15.2MiB	7.94MiB	165.76KiB	0	0.522202	True	True
splunk_hec_route_s3	-2.69MiB	-14.7	100.00%	18.32MiB	2.35MiB	48.83KiB	0	0.128005	15.63MiB	2.42MiB	50.63KiB	0	0.154944	False	False
syslog_loki	-3.35MiB	-23.53	100.00%	14.23MiB	427.33KiB	8.74KiB	0	0.029326	10.88MiB	584.75KiB	11.89KiB	0	0.0524755	False	False
syslog_regex_logs2metric_ddmetrics	-3.08MiB	-25.3	100.00%	12.18MiB	624.09KiB	12.71KiB	0	0.050041	9.1MiB	399.11KiB	8.14KiB	0	0.0428377	False	False
http_pipelines_blackhole_acks	-309.88KiB	-25.6	100.00%	1.18MiB	115.83KiB	2.36KiB	0	0.0956698	900.56KiB	61.05KiB	1.24KiB	0	0.0677743	False	False
syslog_humio_logs	-4.32MiB	-29.35	100.00%	14.72MiB	1.75MiB	36.64KiB	0	0.118964	10.4MiB	1.2MiB	25.22KiB	0	0.115648	False	False
syslog_splunk_hec_logs	-4.7MiB	-29.42	100.00%	15.98MiB	838.77KiB	17.07KiB	0	0.0512529	11.28MiB	490.9KiB	10.03KiB	0	0.042501	False	False
datadog_agent_remap_datadog_logs	-19.85MiB	-31.81	100.00%	62.41MiB	2.52MiB	52.75KiB	0	0.0403686	42.56MiB	2.78MiB	57.95KiB	0	0.0653612	False	False
syslog_log2metric_splunk_hec_metrics	-5.68MiB	-33.11	100.00%	17.16MiB	911.81KiB	18.58KiB	0	0.0518666	11.48MiB	605.27KiB	12.32KiB	0	0.0514709	False	False
datadog_agent_remap_datadog_logs_acks	-20.33MiB	-33.26	100.00%	61.12MiB	3.05MiB	63.78KiB	0	0.0498283	40.79MiB	3.02MiB	62.93KiB	0	0.074097	False	False
http_pipelines_blackhole	-574.76KiB	-33.37	100.00%	1.68MiB	18.57KiB	388.28B	0	0.0107804	1.12MiB	82.42KiB	1.68KiB	0	0.0718009	False	False
http_pipelines_no_grok_blackhole	-5.34MiB	-48.96	100.00%	10.9MiB	132.77KiB	2.71KiB	0	0.0118884	5.57MiB	552.31KiB	11.23KiB	0	0.0969006	False	False
datadog_agent_remap_blackhole_acks	-34.44MiB	-55.85	100.00%	61.67MiB	4.6MiB	95.8KiB	0	0.0745545	27.22MiB	1.14MiB	23.87KiB	0	0.0419289	False	False
datadog_agent_remap_blackhole	-36.02MiB	-56.83	100.00%	63.38MiB	4.22MiB	87.93KiB	0	0.0665733	27.36MiB	1.26MiB	26.26KiB	0	0.0459867	False	False

[github-actions]

Soak Test Results

Baseline: 573b309
Comparison: 7d6ebf0
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

Changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

experiment	Δ mean	Δ mean %	confidence
splunk_hec_route_s3	-1.88MiB	-10.77	100.00%
syslog_loki	-1.7MiB	-12.33	100.00%
http_pipelines_blackhole_acks	-208.78KiB	-17.21	100.00%
syslog_regex_logs2metric_ddmetrics	-2.26MiB	-18.56	100.00%
syslog_splunk_hec_logs	-3.55MiB	-21.92	100.00%
syslog_humio_logs	-3.73MiB	-22.33	100.00%
http_pipelines_blackhole	-400.06KiB	-23.77	100.00%
datadog_agent_remap_datadog_logs	-15.24MiB	-24.9	100.00%
datadog_agent_remap_datadog_logs_acks	-15.55MiB	-25.44	100.00%
syslog_log2metric_splunk_hec_metrics	-4.51MiB	-25.67	100.00%
http_pipelines_no_grok_blackhole	-4.12MiB	-37.84	100.00%
datadog_agent_remap_blackhole_acks	-28.67MiB	-47.43	100.00%
datadog_agent_remap_blackhole	-29.41MiB	-47.94	100.00%

Fine details of change detection per experiment.

experiment	Δ mean	Δ mean %	confidence	baseline mean	baseline stdev	baseline stderr	baseline outlier %	baseline CoV	comparison mean	comparison stdev	comparison stderr	comparison outlier %	comparison CoV	erratic	declared erratic
http_text_to_http_json	2.12MiB	5.85	100.00%	36.23MiB	742.34KiB	15.15KiB	0	0.020008	38.35MiB	814.66KiB	16.63KiB	0	0.020743	False	False
socket_to_socket_blackhole	1.01MiB	4.57	100.00%	22.04MiB	513.79KiB	10.49KiB	0	0.0227649	23.04MiB	321.86KiB	6.57KiB	0	0.0136381	False	False
splunk_hec_to_splunk_hec_logs_acks	20.0KiB	0.08	56.59%	23.74MiB	924.52KiB	18.8KiB	0	0.0380214	23.76MiB	851.45KiB	17.32KiB	0	0.0349877	False	False
splunk_hec_indexer_ack_blackhole	2.83KiB	0.01	9.82%	23.77MiB	807.8KiB	16.44KiB	0	0.0331868	23.77MiB	783.81KiB	15.95KiB	0	0.0321978	False	False
splunk_hec_to_splunk_hec_logs_noack	2.38KiB	0.01	18.88%	23.84MiB	331.45KiB	6.77KiB	0	0.0135741	23.84MiB	357.95KiB	7.31KiB	0	0.014658	False	False
enterprise_http_to_http	-1.6KiB	-0.01	16.04%	23.85MiB	253.75KiB	5.18KiB	0	0.0103895	23.84MiB	293.18KiB	6.0KiB	0	0.0120046	False	False
file_to_blackhole	-51.63KiB	-0.05	41.29%	95.34MiB	3.07MiB	63.65KiB	0	0.0321986	95.29MiB	3.4MiB	70.64KiB	0	0.0356271	False	False
http_to_http_json	-14.63KiB	-0.06	80.74%	23.85MiB	336.01KiB	6.86KiB	0	0.0137571	23.83MiB	434.19KiB	8.89KiB	0	0.0177878	False	False
http_to_http_noack	-64.19KiB	-0.26	99.91%	23.84MiB	255.89KiB	5.24KiB	0	0.0104781	23.78MiB	913.56KiB	18.62KiB	0	0.037507	False	False
fluent_elasticsearch	-308.83KiB	-0.38	100.00%	79.47MiB	53.5KiB	1.08KiB	0	0.000657318	79.17MiB	3.62MiB	74.39KiB	0	0.0456983	False	False
syslog_log2metric_humio_metrics	-624.07KiB	-4.91	100.00%	12.42MiB	552.24KiB	11.27KiB	0	0.0433973	11.81MiB	607.17KiB	12.37KiB	0	0.0501747	False	False
splunk_hec_route_s3	-1.88MiB	-10.77	100.00%	17.46MiB	2.34MiB	48.76KiB	0	0.134055	15.58MiB	2.33MiB	48.74KiB	0	0.149587	False	False
syslog_loki	-1.7MiB	-12.33	100.00%	13.79MiB	598.78KiB	12.26KiB	0	0.0424006	12.09MiB	732.69KiB	14.89KiB	0	0.0591797	False	False
http_to_http_acks	-2.64MiB	-15.29	100.00%	17.28MiB	8.1MiB	169.42KiB	0	0.468688	14.64MiB	7.7MiB	160.91KiB	0	0.526294	True	True
http_pipelines_blackhole_acks	-208.78KiB	-17.21	100.00%	1.18MiB	110.41KiB	2.25KiB	0	0.0910152	1004.11KiB	62.84KiB	1.28KiB	0	0.0625694	False	False
syslog_regex_logs2metric_ddmetrics	-2.26MiB	-18.56	100.00%	12.2MiB	642.65KiB	13.08KiB	0	0.0514182	9.94MiB	491.88KiB	10.03KiB	0	0.0483233	False	False
syslog_splunk_hec_logs	-3.55MiB	-21.92	100.00%	16.2MiB	788.15KiB	16.04KiB	0	0.0475129	12.65MiB	418.95KiB	8.56KiB	0	0.0323465	False	False
syslog_humio_logs	-3.73MiB	-22.33	100.00%	16.7MiB	180.53KiB	3.69KiB	0	0.0105557	12.97MiB	113.72KiB	2.33KiB	0	0.00856074	False	False
http_pipelines_blackhole	-400.06KiB	-23.77	100.00%	1.64MiB	43.45KiB	908.94B	0	0.0258057	1.25MiB	64.79KiB	1.32KiB	0	0.0504795	False	False
datadog_agent_remap_datadog_logs	-15.24MiB	-24.9	100.00%	61.21MiB	440.0KiB	9.01KiB	0	0.00701847	45.97MiB	2.81MiB	58.58KiB	0	0.0611475	False	False
datadog_agent_remap_datadog_logs_acks	-15.55MiB	-25.44	100.00%	61.12MiB	3.07MiB	64.19KiB	0	0.0502304	45.57MiB	3.22MiB	67.0KiB	0	0.0706147	False	False
syslog_log2metric_splunk_hec_metrics	-4.51MiB	-25.67	100.00%	17.58MiB	711.35KiB	14.5KiB	0	0.0395183	13.06MiB	578.53KiB	11.78KiB	0	0.0432385	False	False
http_pipelines_no_grok_blackhole	-4.12MiB	-37.84	100.00%	10.88MiB	289.54KiB	5.91KiB	0	0.0259772	6.76MiB	658.18KiB	13.39KiB	0	0.0949994	False	False
datadog_agent_remap_blackhole_acks	-28.67MiB	-47.43	100.00%	60.46MiB	4.71MiB	98.07KiB	0	0.0778638	31.78MiB	1.62MiB	34.01KiB	0	0.0511019	False	False
datadog_agent_remap_blackhole	-29.41MiB	-47.94	100.00%	61.34MiB	4.03MiB	84.06KiB	0	0.0657264	31.93MiB	1.47MiB	30.63KiB	0	0.0459336	False	False

[jszwedko]

Superseded by #17465