Skip to content

What configuration was used to compare the training performance of oft and lora on the qwen3-30b-a3b model? #3

Description

@deepllz

As shown in the image in the article(https://spherelab.ai/orbit/#qwen3),
Is the training configuration based on https://github.com/Sphere-AI-Lab/orbit/blob/main/examples/high_precision/run-qwen3-30b-a3b-bf16-openr1-oft-b64.sh from the repository? I usually try to reproduce this using Slime, but the results drop after a few dozen training steps. Do you have any suggestions?

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions