Refactors train loop, adds padded batch packer, other fixes#654
Merged
Maxusmusti merged 2 commits intofixed-speed-gpt-oss-support-freezing-fix-lossfrom Sep 15, 2025
Merged
Conversation
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
9813f8a to
e41ea48
Compare
e41ea48 to
0253e48
Compare
0253e48 to
ca6293b
Compare
c1bb315
into
fixed-speed-gpt-oss-support-freezing-fix-loss
1 check passed
RobotSail
added a commit
that referenced
this pull request
Sep 16, 2025
* addition of padded batch packer + simplified train loop * update tests + linting
RobotSail
added a commit
that referenced
this pull request
Sep 16, 2025
* addition of padded batch packer + simplified train loop * update tests + linting
RobotSail
added a commit
that referenced
this pull request
Sep 16, 2025
* addition of padded batch packer + simplified train loop * update tests + linting
Maxusmusti
added a commit
that referenced
this pull request
Sep 17, 2025
* Adding dequantized load support for gpt_oss models Signed-off-by: Mustafa Eyceoz <[email protected]> * Update gpt oss saving with requantization Signed-off-by: Mustafa Eyceoz <[email protected]> * Adjust data processing for gpt format Signed-off-by: Mustafa Eyceoz <[email protected]> * fix for exact quantization algorithm to replicate OpenAI quantized weights * Speedup replicate implementation Signed-off-by: Mustafa Eyceoz <[email protected]> * router freezing for gpt oss Signed-off-by: Mustafa Eyceoz <[email protected]> * Add corrected loss, aux loss support, and batching updates Signed-off-by: Mustafa Eyceoz <[email protected]> * Cleanup unnecessary test files Signed-off-by: Mustafa Eyceoz <[email protected]> * Fix linting and review feedback Signed-off-by: Mustafa Eyceoz <[email protected]> * Add linting skip for mxfp4 import Signed-off-by: Mustafa Eyceoz <[email protected]> * Fix unit tests with mock configs Signed-off-by: Mustafa Eyceoz <[email protected]> * Switch to mini trainer sampler Signed-off-by: Mustafa Eyceoz <[email protected]> * remove dead code + add defaults (#653) * Refactors train loop, adds padded batch packer, other fixes (#654) * addition of padded batch packer + simplified train loop * update tests + linting * fix tests x2 --------- Signed-off-by: Mustafa Eyceoz <[email protected]> Co-authored-by: Nikhil Nayak <[email protected]> Co-authored-by: Oleg Silkin <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a number of improvements:
There are still a few other things to fix but this resolves the bulk of the outstanding items