I never got a Computer Science degree. In college, I majored in Psychology. And as part of the work-study program, I worked in several Psychology labs on campus. It was this experience that made it very very clear to me that I never wanted to be a researcher.
Let me explain… First, there is nothing wrong with being a researcher. It’s just not for me. I saw professors who dedicated THEIR WHOLE LIFE to only one tiny part of research. They just kept doing variations of the same experiment over and over and over again. Writing their findings up for a small number of other researchers in the same field who were the only ones who cared about that same super narrow topic.
Well, fast-forward to today and I’m back to doing a few very narrow experiments on my own in a topic that very very few others care about. For about the past two weeks, I’ve been working with Opus 5.5 on tuning an open source model from HuggingFace to generate pronunciations for words in my Sanskrit Dictionary app. Here is what the process looks like:
But first, the setup…
Just like with my normal daily coding tasks, I naturally started my Claude session work by downloading and running things on my Mac. BIG MISTAKE!!! The models are BIG and the runs are heavy. DO NOT DO THIS haha. It’s slooooow and will fry your Mac.
Instead, I pivoted to using Google Collab. Claude can generate the notebooks for you and you can use the Claude browser plugin agent to navigate the hard parts in the notebook if needed. It was super nice to have the browser agent fix the mistakes that happened on the first run and write a report of what went wrong for the session agent. Another fun tip: Claude can write the prompt for your browser agent for exactly what it needs to do!
Now, you don’t have to use Google Collab. I’m sure there are many other options, including renting your own VM and running this process. But that is what works well for me. I LOVE Google Collab!!! The whole experimentation process cost me only $11 - the Google Pro subscription price! The full run would work on the $60 plan! Just wish they created a CLI tool for it that agents can control directly.
Experiment 1: Which Model? Which Dataset?
HuggingFace has a TON of models. In my case, for my field (Sanskrit) it is a lot more limited. If you have a more broad topic that you’re tuning for, then this part will be a much bigger experiment!
Claude can help generate a golden examples test set from your full dataset to test with your goal in mind. Then it can run experiments with different models and open-source datasets to help you figure out which combination is best. I had Claude generate secret webpages on my website with the results I could test for every set, and keep going from there.
Be creative with Claude and the generated results website to make this part easy for you to work with! I would NOT want to go through every file in Google Docs! For me, the website generated would have experiment samples side-by-side and I could write notes about each version. What I liked, what I didn’t like. My notes were automatically stored in text format at the top of the file, which I could copy-paste into my Claude session to continue from.
Research is TEDIOUS! Having AI simplifying the extra tedious parts for you is seriously a miracle! I still remember my work in the Psychology labs labeling every single thing by hand in an excel sheet!
Experiment 2: Tuning the Model
Once I identified the best combination of model + dataset to work with, I then had Claude run experiments to find the best combination for my use-case.
For example, I identified many mistakes in the Sanskrit pronunciations, and the model would test out how to spell those words in a way that the model is prompted to pronounce it correctly. It would then run this spelling against other words with similar spelling to see if the rule could be applied universally or for certain cases.
Sounds simple, but each run takes a long time. In my case, ~15 minutes. So it’s that long game of waiting for the results only to be disappointed sometimes and having to re-run it all again. At one point, it’s hard to keep faith that this whole thing will work at all!
Ideally, I would have done a full fine-tuning, and I will in the future, but that would require for me to creating my own full training dataset first! I had to keep working within the constraints of the model and training data that was available on HuggingFace. Thank you all the researchers that have made it available publicly!!
Experiment 3: Adjusting the Output
The tuning will only get you so far. Some code processing is also needed. In my case, many voice recordings were generated with static or echo. The code needed to identify that and rerun them in different seeds.
The final code ended up having many “rules” for how to process different words with different spellings, with different punctuation, and identifying and rerunning the bad results.
The Takeaway
OpenA, Anthropic, etc do SUCH a good job at making their models “just work” that we have now started to take them for granted. But just try to download an open source model like DeepSeek and you will be shocked by how bad it is out of the box! SHOCKED!
Behind all that work stands unlimited hours of extremely tedious research work. For me, spending a few days on just a tiny fraction of this was extremely frustrating and tedious. I cannot imagine being a researcher whose job it is to run hundreds and hundreds of these tiny experiments per day. And not to mention the gathering and labelling the data.
I’m extremely grateful for all of those involved who make this technology possible for us to use so easily!
UPDATE: Looks like Google Collab does have a CLI!!! OMG!!! Thanks Taylor Arndt for pointing that out. Just wish I knew about it last week… But better late than never! Will use for my next model tuning!


Loved this! I have done some fine-tuning myself and the waiting game is so real.
Good news on your CLI wish: Google released a Colab CLI in June that agents can control directly. You can spin up a GPU, run your script, and download the results all from the terminal, and it works with Claude Code too: https://developers.googleblog.com/introducing-the-google-colab-cli/
On the Mac side, it really depends on the Mac and the model. Apple's MLX framework lets you do LoRA fine-tuning on smaller models right on Apple silicon, and the more unified memory you have, the bigger you can go. It is not the right fit for every job, but it is great for quick tests before paying for cloud time.
RunPod is another good option and you get a lot of GPU choices. A couple of ways to save money there:
Community Cloud uses GPUs hosted by individual providers instead of RunPod's own data centers. It is usually cheaper, but reliability can vary.
Spot pods are even cheaper, but your run can get interrupted if someone else needs that GPU. If you go that route, have your training save checkpoints often so you can pick up where you left off instead of starting over. With 15 minute runs that might not matter much, but for longer jobs it is a lifesaver.
If you need a run to finish no matter what, pay a little more for an on-demand pod.