Training ops kernels: Speeding up the Llama-based MoE architectures#6734

Draft

RezaYazdaniAminabadi wants to merge 9 commits intomicrosoft:masterfrom RezaYazdaniAminabadi:tops-kernels

+6,248

Commits on Apr 7, 2024

Optimize the fp-dequantizer to get high memory-BW utilization
sfc-gh-reyazda
committed

Commits on Apr 10, 2024

Commits on Nov 4, 2024

Merge branch 'microsoft:master' into master
RezaYazdaniAminabadi
authored

Commits on Nov 8, 2024

Add several kernels to speed up training llama-based MoE model architectures
sfc-gh-reyazda
committed
Merge branch 'master' into tops-kernels
RezaYazdaniAminabadi
authored

Commits on Nov 12, 2024

Merge branch 'master' into tops-kernels
loadams
authored

Commits on Dec 18, 2024

Merge branch 'master' into tops-kernels
loadams
authored