Thank you Natasha. Do you think there's a benefit to switch to OpenRouter when we benefit from Azure's startup credits? (As share by you in a previous post).
I'm worried about Azure's performance.
Plus I heard that OpenRouter offers many free to use models. (As long as you add a little bit of credit top up, that won't be used anyway for free models)
Tarek - from my experience using the Azure model more, it was super rate limited, and I had to reach out to increase the limit... The main thing is to test across models. So if you implement the Azure implementation, then use OpenRouter as fallback, that should work well. If there is an error with Azure, then route to OpenRouter. It's not too hard to do this yourself for 1 - 3 models, but becomes harder when you want to test out many models. So that's where OpenRouter has the benefit.
I see, that's interesting to know. Is there a particular error thrown when rate limited by Azure? Just so that I know it before running into it (I guess I'll see it in production, but it might be too late 😅)
Also another question that came to me. When using OpenRouter against multiple vendors, is there a unified way for non string input? Like images, tools, web search etc.
Thank you Natasha. Do you think there's a benefit to switch to OpenRouter when we benefit from Azure's startup credits? (As share by you in a previous post).
I'm worried about Azure's performance.
Plus I heard that OpenRouter offers many free to use models. (As long as you add a little bit of credit top up, that won't be used anyway for free models)
Tarek - from my experience using the Azure model more, it was super rate limited, and I had to reach out to increase the limit... The main thing is to test across models. So if you implement the Azure implementation, then use OpenRouter as fallback, that should work well. If there is an error with Azure, then route to OpenRouter. It's not too hard to do this yourself for 1 - 3 models, but becomes harder when you want to test out many models. So that's where OpenRouter has the benefit.
I see, that's interesting to know. Is there a particular error thrown when rate limited by Azure? Just so that I know it before running into it (I guess I'll see it in production, but it might be too late 😅)
Also another question that came to me. When using OpenRouter against multiple vendors, is there a unified way for non string input? Like images, tools, web search etc.
For OpenRouter - take a look at their docs. They do support Images, Tool Calling & Web Search: https://openrouter.ai/docs/features/tool-calling - all unified!
That's great! Thanks Natasha 😊
Yes - HTTP 429 (“Too Many Requests”) and the JSON error will include a "type": "rate_limit_exceeded" field.