|
1 | | -# NAME |
2 | | - |
3 | | -Benchmark::DKbench - Perl Benchmark |
4 | | - |
5 | | -# SYNOPSIS |
6 | | - |
7 | | - # Run the suite single-threaded and then multi-threaded on multi-core systems |
8 | | - # Will print scores for the two runs and multi/single thread scalability |
9 | | - dkbench |
10 | | - |
11 | | - # A dual-thread run with times instead of scores |
12 | | - dkbench -j 2 -t |
13 | | - |
14 | | - # If BioPerl is installed, enable the BioPerl benchmarks by downloading Genbank data |
15 | | - dkbench --setup |
16 | | - |
17 | | - # Force install the reference versions of all CPAN modules |
18 | | - setup_dkbench --force |
19 | | - |
20 | | -# DESCRIPTION |
21 | | - |
22 | | -A Perl benchmark suite for general compute, created to evaluate the comparative |
23 | | -performance of systems when running computationally intensive Perl (both pure Perl |
24 | | -and XS) workloads. Runs single and multi-threaded and can be fully customized to |
25 | | -the benchmarks that better suit your own scenario. |
26 | | - |
27 | | -The only non-CPAN libraries required to install/run the suite are libdb, libjpeg |
28 | | -(with headers). There are a few prerequisite CPAN libraries, plus an optional one |
29 | | -([BioPerl](https://metacpan.org/pod/BioPerl), requires libxml on system) which enables extra benchmarks. |
30 | | - |
31 | | -# SCRIPTS |
32 | | - |
33 | | -You should only ever need the main script `dkbench` for the benchmark suite, and, |
34 | | -perhaps `setup_dkbench` can help with setup or standardizing/normalizing your |
35 | | -benchmarking environment. |
36 | | - |
37 | | -## `dkbench` |
38 | | - |
39 | | -The main script that runs the DKbench benchmark suite. If [BioPerl](https://metacpan.org/pod/BioPerl) is installed, |
40 | | -you may want to start your first run with `dkbench --setup`. Otherwise, there are |
41 | | -many options to control number of threads, iterations, which benchmarks to run etc: |
42 | | - |
43 | | - dkbench [options] |
44 | | - |
45 | | - Options: |
46 | | - --threads <i>, -j <i> : Number of benchmark threads (default is 1). |
47 | | - --multi, -m : Multi-threaded using all your CPU cores/threads. |
48 | | - --iter <i>, -i <i> : Number of suite iterations (with min/max/avg at the end). |
49 | | - --include <regex> : Run only benchmarks that match regex. |
50 | | - --exclude <regex> : Do not run benchmarks that match regex. |
51 | | - --time, -t : Report time (sec) instead of score. |
52 | | - --quick, -q : Quick benchmark run (implies -t). |
53 | | - --no_mce : Do not run under MCE::Loop (implies -j 1). |
54 | | - --skip_bio : Skip BioPerl benchmarks. |
55 | | - --skip_timep : Skip Time::Piece benchmark (see benchmark details on why). |
56 | | - --skip_prove : Skip Moose prove benchmark. |
57 | | - --bio_codons : Run optional BioPerl Codons benchmark (does not scale well). |
58 | | - --sleep <i> : Sleep for <i> secs after each benchmark. |
59 | | - --setup : Download the Genbank data to enable the BioPerl tests. |
60 | | - --datapath <path> : Override the path where the expected benchmark data is found. |
61 | | - |
62 | | -The default run (no options) will run all the benchmarks both single-threaded and |
63 | | -multi-threaded (using all detected CPU cores/hyperthreads) and show you scores and |
64 | | -multi vs single threaded scalability. |
65 | | - |
66 | | -The scores are such that a reference CPU (Intel Xeon Platinum 8481C - Sapphire Rapids) |
67 | | -would get 1000 on a single core benchmark run using the default software configuration |
68 | | -(Linux/Perl 5.36.0/ref CPAN modules). |
69 | | - |
70 | | -The multi-thread scalability should approach 100% if each thread runs on a full core |
71 | | -(i.e. no SMT), and the core can maintain the clock speed it had on the single-thread |
72 | | -runs. Note that the overall scalability is an average of the benchmarks that drops |
73 | | -non-scaling outliers (over 2\*stdev less than the mean). |
74 | | - |
75 | | -The suite will report a Pass/Fail per benchmark. A failure may be caused if you have |
76 | | -different CPAN module version installed - this is normal, and you will be warned. |
77 | | - |
78 | | -The suite uses [MCE::Loop](https://metacpan.org/pod/MCE%3A%3ALoop) to run on the desired number of parallel threads, although |
79 | | -there is an option to disable it, which forces a single-thread run. |
80 | | - |
81 | | -## `setup_dkbench` |
82 | | - |
83 | | -Simple installer to check/get the reference versions of CPAN modules and download |
84 | | -the Genbank data file required for the BioPerl benchmarks of the DKbench suite. |
85 | | - |
86 | | -It assumes that you have some software already installed (e.g. `cpanm`), try |
87 | | -`setup_dkbench --help` for more details on that. |
88 | | - |
89 | | - setup_dkbench [--force --sudo --test --data=s --help] |
90 | | - |
91 | | - Options: |
92 | | - --sudo : Will use sudo for cpanm calls. |
93 | | - --force : Will install reference CPAN module versions and re-download the genbank data. |
94 | | - --test : Will run the test suites for the CPAN module (default behaviour is to skip). |
95 | | - --data=s : Data dir path to copy files from. Should not need if you installed DKbench. |
96 | | - --help : Print this help text and exit. |
97 | | - |
98 | | -Running it without any options will fetch the data for the BioPerl tests (similar to |
99 | | -`dkbench --setup`) and use `cpanm` to install any missing libraries. |
100 | | - |
101 | | -Using it with `--force` will install the reference CPAN module versions, including |
102 | | -BioPerl which is not a requirement for DKbench, but enables the BioPerl benchmarks. |
103 | | - |
104 | | -The reference CPAN versions are suggested if you want a fair comparison between |
105 | | -systems and also for the benchmark Pass/Fail results to be reliable. |
106 | | - |
107 | | -# BENCHMARKS |
108 | | - |
109 | | -The suite consists of 19 benchmarks. With a successful installation of the DKbench |
110 | | -package, 18 will run by default, although the `BioPerl Monomers` requires BioPerl |
111 | | -to be installed and Genbank data to be downloaded (`dkbench --setup` can do the |
112 | | -latter). |
113 | | - |
114 | | -On MacOS only, the `Time::Piece` benchmark will be skipped, unless you try a single |
115 | | -thread run without MCE (`--skip_mce`). |
116 | | - |
117 | | -- `Astro` : Calculates precession between random epochs and finds the |
118 | | -constellation for random equatorial coordinates using [Astro::Coord::Precession](https://metacpan.org/pod/Astro%3A%3ACoord%3A%3APrecession) |
119 | | -and [Astro::Coord::Constellations](https://metacpan.org/pod/Astro%3A%3ACoord%3A%3AConstellations) respectively. |
120 | | -- `BioPerl Codons` : Counts codons on a sample bacterial sequence. Requires |
121 | | -[BioPerl](https://metacpan.org/pod/BioPerl) to be installed. |
122 | | -This test does not scale well on multiple threads, so is disabled by default (use |
123 | | -`--bio_codons`) option. Requires data fetched using the `--setup` option. |
124 | | -- `BioPerl Monomers` : Counts monomers on 500 sample bacterial sequences using |
125 | | -[BioPerl](https://metacpan.org/pod/BioPerl) (which needs to be installed). Requires data fetched using the `--setup` |
126 | | -option. |
127 | | -- `CSS::Inliner` : Inlines CSS on 2 sample wiki pages using [CSS::Inliner](https://metacpan.org/pod/CSS%3A%3AInliner). |
128 | | -- `Crypt::JWT` : Creates large JSON Web Tokens with RSA and EC crypto keys |
129 | | -using [Crypt::JWT](https://metacpan.org/pod/Crypt%3A%3AJWT). |
130 | | -- `DateTime` : Creates and manipulates [DateTime](https://metacpan.org/pod/DateTime) objects. |
131 | | -- `Digest` : Creates MD5, SH1 and SHA-512 digests of a large string. |
132 | | -- `Encode` : Encodes/decodes large strings from/to UTF-8/16, cp-1252. |
133 | | -- `HTML::FormatText` : Converts HTML to text for 2 sample wiki pages using |
134 | | -[HTML::FormatText](https://metacpan.org/pod/HTML%3A%3AFormatText). |
135 | | -- `Imager` : Loads a sample image and performs edits/manipulations with |
136 | | -[Imager](https://metacpan.org/pod/Imager), including filters like gaussian, unsharp mask, mandelbrot. |
137 | | -- `JSON::XS` : Encodes/decodes random data structures to/from JSON using |
138 | | -[JSON::XS](https://metacpan.org/pod/JSON%3A%3AXS). |
139 | | -- `Math::DCT` : Does 8x8, 18x18 and 32x32 DCT transforms with [Math::DCT](https://metacpan.org/pod/Math%3A%3ADCT). |
140 | | -- `Math::MatrixReal` : Performs various manipulations on [Math::MatrixReal](https://metacpan.org/pod/Math%3A%3AMatrixReal) |
141 | | -matrices. |
142 | | -- `Moose` : Creates [Moose](https://metacpan.org/pod/Moose) objects. |
143 | | -- `Moose prove` : Runs 110 tests from the Moose 2.2201 test suite. The least |
144 | | -CPU-intensive test, most of the time will be spent loading the interpreter and |
145 | | -the Moose module for each test, which is behaviour representative of how a Perl |
146 | | -test suite runs by default. |
147 | | -- `Primes` : Calculates all primes up to 7.5 million. Small number with |
148 | | -repeat was chosen to keep low memory (this is a pure Perl function no Math libraries). |
149 | | -- `Regex/Subst` : Concatenates 3 wiki pages into a byte string then matches |
150 | | -3 typical regexes (for names, emails, URIs), replaces html tags with their contents |
151 | | -(starting with the innermost) and does calls subst a few times. |
152 | | -- `Regex/Subst utf8` : Exactly the same as `Regex/Subst`, but reads into |
153 | | -a utf8 string. Perl version can make a big difference, as Unicode behaviour has |
154 | | -changed (old Perl versions are faster but less strict in general). |
155 | | -- `Time::Piece` : Creates and manipulates/converts Time::Piece objects. It |
156 | | -is disabled by default on MacOS (unless `--no_mce` is specified), as it runs |
157 | | -extremely slow when forked on this platform. In general you may want to skip this |
158 | | -benchmark (`--skip_timep`) if you are comparing different OS platforms. |
159 | | - |
160 | | -# EXPORTED FUNCTIONS |
161 | | - |
162 | | -You will normally not use the Benchmark::DKbench module itself, but here are the |
163 | | -exported functions that the `dkbench` script uses for reference: |
164 | | - |
165 | | -## `system_identity` |
166 | | - |
167 | | - my $cores = system_identity(); |
168 | | - |
169 | | -Prints out software/hardware configuration and returns then number of cores detected. |
170 | | - |
171 | | -## `suite_run` |
172 | | - |
173 | | - my %stats = suite_run(\%options); |
174 | | - |
175 | | -Runs the benchmark suite given the `%options` and prints results. Returns a hash |
176 | | -with run stats. |
177 | | - |
178 | | -## `calc_scalability` |
179 | | - |
180 | | - calc_scalability(\%options, \%stat_single, \%stat_multi); |
181 | | - |
182 | | -Given the `%stat_single` results of a single-threaded `suite_run` and `%stat_multi` |
183 | | -results of a multi-threaded run, will calculate and print the multi-thread scalability. |
184 | | - |
185 | | -# NOTES |
186 | | - |
187 | | -The benchmark suite was created to compare the performance of various cloud offerings. |
188 | | -You can see the [original perl blog post](http://blogs.perl.org/users/dimitrios_kechagias/2022/03/cloud-provider-performance-comparison-gcp-aws-azure-perl.html) |
189 | | -as well as the [2023 follow-up](https://dev.to/dkechag/cloud-vm-performance-value-comparison-2023-perl-more-1kpp). |
190 | | - |
191 | | -The benchmarks for the first version were more tuned to what I would expect to run |
192 | | -on the servers I was testing, to choose the optimal types for the company I was |
193 | | -working for. The second version has expanded a bit over that and is friendlier to |
194 | | -use. |
195 | | - |
196 | | -# AUTHOR |
197 | | - |
198 | | -Dimitrios Kechagias, `<dkechag at cpan.org>` |
199 | | - |
200 | | -# BUGS |
201 | | - |
202 | | -Please report any bugs or feature requests either on [GitHub](https://github.com/dkechag/Benchmark-DKbench) (preferred), or on RT (via the email |
203 | | -`bug-Benchmark-DKbench at rt.cpan.org` or [web interface](https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Benchmark-DKbench)). |
204 | | - |
205 | | -I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. |
206 | | - |
207 | | -# GIT |
208 | | - |
209 | | -[https://github.com/dkechag/Benchmark-DKbench](https://github.com/dkechag/Benchmark-DKbench) |
210 | | - |
211 | | -# LICENSE AND COPYRIGHT |
212 | | - |
213 | | -This software is copyright (c) 2021-2023 by Dimitrios Kechagias. |
214 | | - |
215 | | -This is free software; you can redistribute it and/or modify it under |
216 | | -the same terms as the Perl 5 programming language system itself. |
0 commit comments