oneDNN BatchNorm + Act fusion pass.

[arogowie-intel]

PR types

New features

PR changes

Others

Describe

This PR adds oneDNN pass for BatchNorm + Act function fusion. Currently only ReLU is supported as activation function.

Performance improvement is around 2.8% - benchmarked on pvanet_ocr model using Xeon 6148 machine. Run with 40 threads.

model	avg running time (s)	relative speedup
base fp32	0,473066	1x
optimized fp32	0,150530	3,143x
optimized fp32 & bn+act	0,146477	1,028x

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[arogowie-intel]

@jczaja @wozna @wojtuss Please take a look and review.

[jczaja]

LGTM

[arogowie-intel]

@luotao1 Could you please help. I see following error occur, although it is not related to my changes:
PR-CI-Windows-OPENBLAS: Line 5993: 2020-10-22 02:31:29 LINK : fatal error LNK1104: cannot open file 'C:\home\workspace\Paddle\build\paddle\fluid\pybind\Release\op_function_generator.exe' [C:\home\workspace\Paddle\build\paddle\fluid\pybind\op_function_generator.vcxproj]

Moreover in PR-CI-Coverage and PR-CI-Windows i get AssertionError: We only support 'data()' in static graph mode, please call 'paddle.enable_static()' to enter static graph mode., although I have called enable_static() here

Finally in PR-CI-Approval I see complaint about the usage of PADDLE_ENFORCE* macro in these lines. I have specified the error type and provided clear, long description. Maybe this is a problem with formatting? However clang-format has passed.

[luotao1]

• PR-CI-Windows-OPENBLAS: you can rerun it, it's a random failure
• PR-CI-Coverage and PR-CI-Windows: Maybe you can refer to other unit-tests like python/paddle/fluid/contrib/slim/tests/test_graph.py

import paddle
import paddle.fluid as fluid
from paddle.fluid.framework import IrGraph
from paddle.fluid import core

paddle.enable_static()
...

• PR-CI-Approval: The check rule is not robust, I will approve it if other CIs all passed.

[arogowie-intel]

@luotao1 All CI-checks (except approval) passed. Could you review please?

[wojtuss]

LGTM

[luotao1]

LGTM