Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge performance improvements from zfex #122

Open
hacklschorsch opened this issue Mar 6, 2025 · 2 comments
Open

Merge performance improvements from zfex #122

hacklschorsch opened this issue Mar 6, 2025 · 2 comments

Comments

@hacklschorsch
Copy link
Member

zfex (also on PyPI) is a performance-focused fork of zfec exploiting SIMD for both, Intel and ARM, with impressive results:

Legacy zfec had both results slightly above 50 MB/sec. zfex in all cases ran faster, achieving best performance with -DZFEX_UNROLL_ADDMUL_SIMD=4 unrolling, giving almost 6-fold speed-up.

Benchmark plot

Having an automatic and seamless way to pick faster versions of the algorithm if the hardware capabilities are available would be really neat.

@sajith
Copy link
Member

sajith commented Mar 20, 2025

It is fantastic that zfex exists and zfex contributors have been able to achieve all those performance gains. It however comes at a cost of some additional complexity and a bigger maintenance overhead. Take a look at https://github.com/WojciechMigda/zfex/blob/main/zfex/zfex.c, for example, and decide for yourself if you really want to add those changes to zfec.

The zfex fork is a result of our refusal to merge hand-rolled assembly to zfec: see #71.

I think there is room for both zfec and zfex in the world: zfex can courageously make all the performance improvements that they can, and zfec can remain simpler and more conservative. There is value in both approaches.

@sajith
Copy link
Member

sajith commented Mar 20, 2025

The zfex fork is a result of our refusal to merge hand-rolled assembly to zfec: see #71.

Well, to be really accurate, it was our inability (mine really, and other folks' unavailability) to review the code, not outright refusal, that caused the fork. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants