I'm Aman Kumar. Most Ruby developers I talk to hit the same wall when they move a Rails app off api.openai.com and onto another OpenAI-compatible server. They copy a base URL from the docs, and either every request 404s or the errors come back with no useful detail. Usually the cause is that there are two popular Ruby clients, and they handle the base URL and errors very differently.
This post covers both: the community ruby-openai gem (8.3 at the time of writing) and OpenAI's official openai gem (0.103). I tested everything below against a local mock server, so the paths and exception classes are what the gems actually do, not what I remember. My examples use APIClaw, an OpenAI-compatible gateway I build, so weigh that accordingly. Nothing here is specific to it. The same code works with vLLM, LiteLLM, Ollama's /v1 endpoint or any server that implements /v1/chat/completions.
This catches people out when they try both. The two gems are required the same way, with require "openai", and both define OpenAI::Client. With both in one Gemfile, which one you get depends on load order, and the constructor arguments are different. Pick one per app.
My rough rule: ruby-openai is a thin wrapper over Faraday that returns plain hashes. It's easy to drop into an existing codebase and easy to add middleware to. The official gem returns typed objects, retries for you and reads the standard environment variables. For a new project I lean towards the official one. For an app that already uses ruby-openai, switching rarely pays off.
# Gemfile: gem "ruby-openai"
require "openai"
client = OpenAI::Client.new(
access_token: ENV.fetch("LLM_API_KEY"),
uri_base: ENV.fetch("LLM_BASE_URL", "https://apiclaw.biz/v1"),
request_timeout: 60,
log_errors: true
)
response = client.chat(parameters: {
model: ENV.fetch("LLM_MODEL"),
messages: [{ role: "user", content: "Say hello in one word." }]
})
puts response.dig("choices", 0, "message", "content")
Unlike most SDKs, ruby-openai doesn't read OPENAI_API_KEY or OPENAI_BASE_URL on its own. You pass values in, or set them once in an initializer with OpenAI.configure { |config| config.uri_base = ... }.
How it builds the URL. The default uri_base is https://api.openai.com/ with no /v1, and the gem adds the version segment itself. It appends v1 unless the string you passed already contains v1. In practice that's forgiving. I tried https://host, https://host/, https://host/v1 and https://host/v1/, and all four ended up at /v1/chat/completions.
The catch is that it's a plain substring check. If your base URL has v1 somewhere else in it, say a proxy path like /gw-v1x/, the gem decides the version is already there. It then posts to /gw-v1x/chat/completions and gets a 404. Keep the base URL to the host plus /v1 and you won't meet this.
Timeouts. request_timeout defaults to 120 seconds. For a web request that's long, so I set it lower and let slow jobs run in a background worker instead.
Errors. Turn on log_errors: true. By default the gem raises a Faraday exception, and the response body, which is where the server explains what went wrong, isn't printed anywhere. With logging on you see the error JSON, and you can rescue the specific classes:
begin
client.chat(parameters: { model: model, messages: messages })
rescue Faraday::UnauthorizedError
# 401: wrong key, or a key sent to the wrong host
rescue Faraday::ResourceNotFound => e
# 404: bad path, or a model ID the server doesn't know
warn e.response[:body].inspect
rescue Faraday::TooManyRequestsError
# 429: back off
rescue Faraday::TimeoutError
# request_timeout passed
end
e.response[:body] is already parsed into a hash, because the gem adds Faraday's JSON response middleware.
Retries. ruby-openai doesn't retry anything. It does accept a block that gets the Faraday connection, so you can add the faraday-retry gem:
require "faraday/retry"
client = OpenAI::Client.new(access_token: key, uri_base: base) do |f|
f.request :retry, max: 2, interval: 0.5, backoff_factor: 2,
retry_statuses: [429, 502, 503], methods: %i[post]
end
I checked this against a server that always returns 429. It made three requests (one plus two retries) and then raised Faraday::TooManyRequestsError. Note methods: %i[post]: by default faraday-retry only retries idempotent methods, and every chat call is a POST. I don't use this client for streaming calls, because a retry halfway through a stream would repeat output the user has already seen.
Streaming. Pass a proc as stream. The gem sets stream: true for you and hands the proc each parsed chunk:
client.chat(parameters: {
model: model,
messages: messages,
stream_options: { include_usage: true },
stream: proc do |chunk, _bytesize|
print chunk.dig("choices", 0, "delta", "content")
puts "\ntokens: #{chunk['usage']['total_tokens']}" if chunk["usage"]
end
})
stream_options: { include_usage: true } asks for a final chunk with token counts. Not every compatible server sends it, so treat usage as optional. Also note that the last chunk can have an empty choices array, which is why I use dig.
# Gemfile: gem "openai"
require "openai"
client = OpenAI::Client.new(
api_key: ENV.fetch("LLM_API_KEY"),
base_url: ENV.fetch("LLM_BASE_URL", "https://apiclaw.biz/v1"),
timeout: 60,
max_retries: 2
)
completion = client.chat.completions.create(
model: ENV.fetch("LLM_MODEL"),
messages: [{ role: "user", content: "Say hello in one word." }]
)
puts completion.choices.first.message.content
If you leave out api_key and base_url, it reads OPENAI_API_KEY and OPENAI_BASE_URL. That's convenient, but I name things explicitly when an app talks to more than one provider.
The base URL must include /v1. This is the big difference. The official gem adds only the endpoint path to whatever you give it. With https://host it posted to /chat/completions and raised OpenAI::Errors::NotFoundError. With https://host/v1 or https://host/v1/ it worked. So a base URL without /v1 that works fine in ruby-openai will 404 here. This is the most common reason a migration between the two "suddenly breaks".
Defaults worth knowing. The timeout is 600 seconds and max_retries is 2. It retried a 429 on its own (two requests with max_retries: 1) before raising. If you also wrap calls in your own retry loop, or run them in a job queue that retries, the attempts multiply, so pick one layer to own retries.
Errors are typed, and each status error carries status and the parsed body:
begin
client.chat.completions.create(model: model, messages: messages)
rescue OpenAI::Errors::AuthenticationError
# 401
rescue OpenAI::Errors::NotFoundError => e
warn e.body.inspect # usually names the bad model or path
rescue OpenAI::Errors::RateLimitError
# 429, after the built-in retries ran out
rescue OpenAI::Errors::APITimeoutError
# timeout passed
rescue OpenAI::Errors::APIStatusError => e
warn "#{e.status}: #{e.body.inspect}"
end
APITimeoutError is a subclass of APIConnectionError, so rescue it first if you want to treat timeouts on their own.
Streaming uses stream_raw, which returns an enumerable of chunk objects:
stream = client.chat.completions.stream_raw(
model: model,
messages: messages,
stream_options: { include_usage: true }
)
stream.each do |chunk|
print chunk.choices.first&.delta&.content
puts "\ntokens: #{chunk.usage.total_tokens}" if chunk.usage
end
Extra parameters. Some servers accept fields that aren't in OpenAI's schema, such as top_k. The official gem types its parameters, so send those through request_options, which also takes a per-call timeout and retry count:
client.chat.completions.create(
model: model,
messages: messages,
request_options: { extra_body: { top_k: 20 }, timeout: 30, max_retries: 0 }
)
In ruby-openai you just add the key to the parameters hash, since it sends the hash as it is.
The other failure I see all the time has nothing to do with Ruby: a guessed model ID. Every provider names models a little differently, and a wrong ID usually comes back as a 404 that looks like a path problem. Before writing anything else, I list what the key can see:
# ruby-openai
client.models.list["data"].each { |m| puts m["id"] }
# official gem
client.models.list.auto_paging_each { |m| puts m.id }
Then I copy the ID exactly into config. Keeping the model in an environment variable (LLM_MODEL above) means switching models is a deploy setting, not a code change.
/v1. The official gem needs it. ruby-openai tolerates leaving it out but trips on a path that contains v1 elsewhere.models.list, not typed from memory.log_errors: true in ruby-openai, or rescue APIStatusError and log body in the official gem.That covers nearly every Ruby integration problem I've debugged against OpenAI-compatible servers. Both gems work well once the base URL and the model ID are right.
Disclosure: I build APIClaw, the OpenAI-compatible gateway used in these examples. Everything here applies to any compatible endpoint.