Small optimizations#768

Open

rhenry-nv wants to merge 14 commits intomarian-nmt:masterfrom rhenry-nv:small_optimizations

+188-62

Commits on Dec 2, 2020

Removed memcpy when batchA == batchB for batched matrix multiplication. Reduced the memcpys from 3 to 1 for the general case.
rhenry-nv
committed
Issue matrix multiplications in cublas and cusparse to the per thread default stream. This have issue each thread issue calls to their own stream instead of the global default stream when marian is…
rhenry-nv
committed
Reduced the number of times indices need to be copied to the GPU
rhenry-nv
committed
Adds licenses to files changed
rhenry-nv
committed
Updates licences
rhenry-nv
committed
Rounds input batches to a multiple of 8 when the gpu backend is being used for better tensor core usage.
rhenry-nv
committed
updates changelog
rhenry-nv
committed

Commits on Dec 15, 2020

Commits on Mar 5, 2021

Merge remote-tracking branch 'public/master' into small_optimizations
rhenry-nv
committed