language-model-adventures

Adventures with Language Models

1. QLoRA vs OpenGPT on Colab

Notes:

⚠ VERY EXPERIMENTAL ⚠

Initial attempts to train an LLM using QLoRA on a free T4 GPU on Colab along the lines of the OpenGPT approach.

Borrowing heavily from and combining the following:

• https://github.com/CogStack/OpenGPT
• https://github.com/artidoro/qlora

Major thanks to the developers of both.