Scratching My Own Itch

Every tool I build at Shahriar Labs starts with a personal frustration. For freelm, the frustration was simple: I was tired of updating API keys.

I build a lot of AI agents. To keep costs down during development, I would use free tiers. I'd use Groq for its speed, Google AI Studio because Gemini 1.5 is incredibly smart and generous, and OpenRouter for access to open-source models.

But managing this was a nightmare.

• My scripts would crash when Groq hit its RPM limit.
• I had to rewrite my API calling logic every time I switched from the OpenAI SDK to the Google SDK.
• Free models on OpenRouter would appear and disappear without warning, breaking my hardcoded model strings.

The Vision for FreeLM

I realized I didn't need another LLM library. I needed a Load Balancer.

I wanted a single client that:

1. Talked the OpenAI Protocol: Because every tool (LangChain, LlamaIndex, Vercel AI SDK) already speaks it.
2. Abstracted the Models: I didn't want to care if the model was llama-3-8b or gemma-7b. I just wanted to ask for a fast model or a large model.
3. Never Crashed: If one provider died, it should seamlessly pivot to another.

Thus, freelm was born.

From Python to Node.js

I originally wrote freelm in Python because that's where most of my backend AI orchestrations live. It quickly became the backbone of several of my side projects.

But as I built more full-stack Next.js applications, I found myself missing the resilience of freelm in my TypeScript environments.

So, I ported it. freelm is now a first-class citizen in the JavaScript ecosystem.

It has zero runtime dependencies (it uses the built-in fetch API), making it incredibly lightweight for both Node.js servers and Edge functions.

You can grab it on npm and PyPI.

The Open Source Commitment

I open-sourced freelm because I believe experimentation in AI should be accessible to everyone, regardless of their budget. By aggregating the generous free tiers provided by companies like Google, Groq, and OpenRouter, developers anywhere in the world can build powerful applications without a credit card.

Check out the source code on GitHub, and if it saves you a few dollars, drop a star!