Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
Hoglet-33 
posted an update 2 days ago
Post
6120
Hey everyone! I got sidetracked from my main projects and decided to test out the BananaAll app and see if I could make a small model not regress too much during SFT. Here is what happened:

The base model I chose was BananaMind/BananaMind-2.1-Pico-Preview, and the dataset I used was SupraLabs/SupraThink-Dataset-500x

I trained for 5 whole steps using a LoRA adapter.

Results:
A model that scores better on some benchmarks and worse on others, and still lacks most general capabilities.

You can find the model here: Hoglet-33/Hogleto

Credits:

- Thank you to @Banaxi-Tech for the BananaAll app (works perfectly on Windows and CPU)
- GPT-6 Sol for knowing how to merge some confusing files created by the app
- Myself for the idea
- Someone else somewhere who might have contributed to some of my ideas and might in the future
- And readers like you!
Bc-AI

@Hoglet-33 nice! I think small models cannot meaningfully learn to perform CoT tho, i once tried a ton of fancy stuff like SFT, GRPO, DPO even a custom tag loss

·
Hoglet-33

yeah i know, the base model doesnt even support those tokens, but it was a dataset i could remember easily so thats what i did
Bc-AI

@Hoglet-33 how did you find Orion? Also I recommend the SFT one it is better at both base and SFT

·
Hoglet-33

the Orion in Refract Labs? I let HF have you and Refract Labs stuff show up in feed plus notifications if thats what your asking, plus pinging me sends a notification. i havnt tested it yet though because of this sidetrack, and i accidentally broke a few things in my code base so once i get all of that sorted i will def test it
In this post