What is Bring Your Own Key (BYOK) via OpenRouter?
BYOK allows you to use AI models that are not natively supported in Afforai via an encrypted OpenRouter API key (we'll never have access to your API key.)
How to set up BYOK on Afforai?
To use Bring Your Own Key in Afforai, go into settings on the ‘custom’ AI model, and press ‘connect to OpenRouter’.
Proceed With Caution:
You'll no longer be using AI services covered by Afforai. You will incur API costs from your OpenRouter account.
I recommend setting a spend limit to prevent overspending.
With BYOK, we're not responsible for OpenRouter-related issues.
How to customize BYOK on Afforai?
After you connect your OpenRouter key, you will see a list of options to customize OpenRouter. Those options are:
BYOK Mode
BYOK Mode
This is the high-level option you can set on how you would like your OpenRouter key to be used. These options are:
Disabled: Your key is still connected to Afforai but won't be used
Afforai Pick: You will use the same models as Afforai's system
Custom: For users who want to choose their own model(s)
OpenRouter's Default: The model will be chosen via OpenRouter UI. We don't recommend this option as the users lack the ability to choose different models for different operations.
Query Model
Query Model
This is the model that oversees answering questions. Users can choose models with a larger context window, such as GPT4-32k or Claude.
Citation Model
Citation Model
This is the model that oversees providing answer citations. We limit the model selection to exclude GPT3.5-16k and GPT4-32k, as the model only requires a 4k context window. Afforai internally uses GPT3.5 for this model.
Indexing Model
Indexing Model
This is the model that oversees indexing documents and retrieving context during queries. This model should be treated as the assistant model to the query model, as such high performance model such as GPT-4 might not be necessary.
Users should be cautious about using expensive models for the Indexing model since uploading files will consume a lot of computation. Afforai internally uses GPT-3.5 for this model.
A few more tips before you configure your BYOK:
Start with Afforai's choice to get a benchmark of how your key is used
If you want to choose your own model, we recommend changing only the Query Model and leaving the rest as GPT-3.5
If you want to experiment greatly, we recommend going for the cheapest options possible for each model, then slowly increase model capability until you find a balance between affordability and performance.
Note: I highly recommend selecting models with at least 16,000 tokens for the context window. Models below are recommended to maintain AI performance.
Choosing models with fewer than 16,000 tokens will impact the quality of AI output and result in citations not being generated due to rate limits.
How to manage your OpenRouter key?
To manage your API key, you can visit the OpenRouter dashboard. You will have the ability to manage your key settings, set spending limits, and monitor your API activities. This allows you to have full control and visibility over your key usage.
In the OpenRouter settings, you will have the option to select the AI model that you prefer.
