I'm Aman Kumar. I spend a lot of time in notebooks, and the %%ai cell magic from Jupyter AI is the quickest way I know to ask a model about a dataframe without leaving the cell I'm in. The catch comes when the model isn't on api.openai.com. Jupyter AI 3 changed how the magics work, and most of the guides you'll find still describe the 2.x openai-chat-custom provider, which isn't there any more.
This post covers the 3.x magics as they ship today: jupyter-ai 3.2.0 with the magics extra, which pulls in jupyter-ai-magic-commands 0.0.4, jupyter-ai-litellm 0.1.0 and LiteLLM 1.105. I ran everything below in IPython 9.17 against a small local mock server that logs each request's path, headers and JSON body, so the results are what the code actually sends. My examples use APIClaw, an OpenAI-compatible gateway I build. Nothing here is specific to it, and the same setup works with vLLM, LiteLLM's proxy, Ollama's /v1 endpoint or any server that implements /v1/chat/completions.
In Jupyter AI 3 the magics are an optional extra, not part of the base package:
pip install "jupyter-ai[magics]"
Then load the extension in a notebook cell:
%load_ext jupyter_ai_magic_commands
Every model call goes through litellm.completion(). That matters, because it's LiteLLM's rules for model names, base URLs and keys that decide where your request goes.
My first try was the obvious one:
%%ai openai/YOUR_MODEL_ID --api-base https://apiclaw.biz/v1
Summarise what a left join does in two sentences.
That prints Model ID 'openai/YOUR_MODEL_ID' is not a known model or alias and sends nothing. The magic checks the model ID against a fixed list of about 3,400 LiteLLM model names before it calls anything, and --api-base doesn't change that. A model that only exists on your gateway will never be on the list.
The fix is an alias, registered with the %ai line magic:
%ai alias claw openai/YOUR_MODEL_ID --api-base https://apiclaw.biz/v1
%%ai claw
Summarise what a left join does in two sentences.
Two details about the target. First, it needs the openai/ prefix. That prefix tells LiteLLM to use its OpenAI-compatible client. Without it, LiteLLM tries to guess the provider from the name and fails with LLM Provider NOT provided. LiteLLM strips the prefix before sending, so the body carries "model": "YOUR_MODEL_ID". Second, only the first segment is stripped. If your gateway's model ID has its own slash, like deepseek/deepseek-chat, write openai/deepseek/deepseek-chat and the server receives deepseek/deepseek-chat, which is what you want.
Keep /v1 in the base URL. LiteLLM appends /chat/completions to whatever you pass, so https://apiclaw.biz without /v1 becomes POST /chat/completions, which my mock (like most servers) answered with a 404.
Both %%ai and %ai alias accept --api-key-name, described as the name of an environment variable in your .env file that holds the key. So I put APICLAW_API_KEY=... in .env and registered the alias with --api-key-name APICLAW_API_KEY.
It didn't do what the name suggests. With no OPENAI_API_KEY set, every call failed with LiteLLM's Missing credentials error. With OPENAI_API_KEY set to something else, the call went through, but the mock showed this:
Authorization: Bearer <value of OPENAI_API_KEY>
body: {"model": "YOUR_MODEL_ID", "messages": [...], "api_key_name": "<value of APICLAW_API_KEY>"}
In this version the magic looks up the variable correctly but hands its value to LiteLLM as a keyword called api_key_name instead of api_key. LiteLLM doesn't treat that as a credential, so it passes it through as an extra field in the JSON body. Your key ends up in the request payload, where gateways, proxies and request loggers can record it, and authentication silently uses a different variable. A strict server may also reject the unknown field.
Until that's fixed, what worked for me was leaving --api-key-name out entirely and putting the gateway key in OPENAI_API_KEY. The extension re-reads .env from the kernel's working directory on every %ai call, with override on, so a .env next to the notebook is enough:
# .env next to the notebook; keep it out of git
OPENAI_API_KEY=your-gateway-key
The trade-off is that OPENAI_API_KEY now means your gateway inside that kernel. If you also call api.openai.com directly from the same notebook, pass that key explicitly in your own code. LiteLLM also reads OPENAI_BASE_URL, and an alias with no --api-base picked it up in my test. I still prefer the explicit --api-base, so the alias says where it points. If you already sent requests with --api-key-name to a server whose logs other people can read, rotating that key is cheap insurance.
Aliases created with %ai alias live in memory and disappear on restart. To make them permanent, set them in your IPython config (~/.ipython/profile_default/ipython_config.py):
c = get_config()
c.InteractiveShellApp.extensions = ["jupyter_ai_magic_commands"]
c.AiMagics.initial_aliases = {
"claw": {
"target": "openai/YOUR_MODEL_ID",
"api_base": "https://apiclaw.biz/v1",
"api_key_name": None,
}
}
c.AiMagics.initial_language_model = "claw"
c.AiMagics.max_history = 2
With initial_language_model set, a bare %%ai with no model ID uses the alias. Keep the api_key_name key in each entry and set it to None. The code reads it directly, and leaving it out made every call to that alias fail with KeyError: 'api_key_name'.
%ai update is listed as a subcommand, but in 0.0.4 it raises an error telling you to use the cell magic. To change an alias, run %ai dealias claw and register it again.
Every %%ai call sends the previous exchanges along with the new prompt. With the default max_history = 2, the mock saw 1, 3, 5, 5, 5 messages on successive calls. That's the last two prompt and answer pairs plus the new prompt. The history is shared across aliases too, so a question to one model carries answers from another. %ai reset clears it, and the next call went back to a single message.
I tried max_history = 0 to turn history off for one-shot cells. It does the opposite. The slice the code uses, transcript[-2 * max_history:], becomes transcript[-0:], which in Python is the whole list. My mock saw 1, 3, 5, 7, 9 messages, growing on every call. On a flat-rate plan that's slower responses. On a per-token plan it's a bill that grows with every cell. Use max_history = 1 if you want the smallest bounded history. With 1, the mock saw 1, 3, 3, 3. Or call %ai reset when you start a new train of thought.
Curly braces are template fields. The cell is run through str.format_map() with your notebook's variables, so {df_name} becomes the value of df_name. Unknown names are left as they are. A literal JSON snippet like {"a": 1} raises ValueError: Space not allowed in string format specifier. Double the braces ({{"a": 1}}) and the model receives {"a": 1}.
-m does nothing yet. The --model-parameters option parses your JSON and then drops it. -m '{"temperature": 0, "max_tokens": 50}' produced a request body with neither field. If you need sampling settings, call LiteLLM or the OpenAI SDK directly for that cell.
Retries and errors. A 429 from the server turned into three POSTs within about a second (the OpenAI SDK's default of two retries), then a printed RateLimitError. API errors like that are printed and returned as a string rather than raised, so a failed model call doesn't stop "Run All". The brace error above and a broken alias do raise, and those will stop it.
-f code inserts a new cell. The response is stripped of its code fence and placed in a new cell below. Read it before you run it.
pip install "jupyter-ai[magics]".openai/ target and an explicit base URL ending in /v1, defined in ipython_config.py.OPENAI_API_KEY via .env, never --api-key-name on 0.0.4.max_history of 1 or 2, never 0, and %ai reset between topics.These packages are moving quickly, so check the changelog for jupyter-ai-magic-commands before you rely on any of the bugs above still being there. The alias and base URL rules come from LiteLLM and will probably stay the same.
Disclosure: I build APIClaw, the OpenAI-compatible gateway in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.