[PHI] feat: support gqa for blha #64640
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
29 tasks
|
FYI：更新了单测，在GQA模式下的blha RoPE 还有些bug，正在debuging。其他的测试本地都已通过～ 现在单测是把GQA和MHA的分开写了，尝试过把GQA和MHA的单测放到一个test里。但是合并写时，势必要执行两次以下的代码。会导致结果错误，如果要再设置两个seq_lens_decoder。在一个test里的变量太多。 self.seq_lens_decoder[:] = self.seq_lens_encoder
self.seq_lens_encoder[:] = 0
self.seq_lens_this_time[:] = 1因此现在选择了把GQA的单测和MHA的单测分开写 |
|
FYI: 把gqa的单测和mha的单测拆开了。并在本地跑过了test |
carryyu
approved these changes
Jun 4, 2024
|
Sorry to inform you that 5dd9b13's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Category
Others
PR Types
Improvements
Description
增强blha，使其支持GQA