Private Model Gateway

AI agents: fetch https://llm.hisavage.com/docs.md (plain markdown, no auth, no JS). Index: https://llm.hisavage.com/llms.txt

This is a private model, served through an OpenAI-compatible API. It has three tiers: default (balanced, general purpose), smart (strongest reasoning) and fast (quickest, lightest). There are also pinned versions of each tier. Any client or SDK built for the OpenAI Chat Completions API works if you change the base URL and the API key.

• Base URL: https://llm.hisavage.com/v1
• Auth header: Authorization: Bearer <YOUR_API_KEY> (keys are issued by the gateway owner; never put a key in code you share)
• Raw markdown of this page (for agents): https://llm.hisavage.com/docs.md

Instructions for AI agents

1. Use the OpenAI Chat Completions format against the base URL above. Read the key from an env var such as GATEWAY_API_KEY.
2. Pick a model from the table below. If you are asked for a Claude model, use its mapped neutral name from the Claude equivalent column (for example "Claude Sonnet 4.5" → default-4.5, "Claude Opus" → smart). If you are asked for an OpenAI model, use its mapped neutral name from the OpenAI equivalent column (for example "gpt-4o-mini" → fast, "gpt-5" → default, "o3" → smart). If unsure, use default.
3. If you're unsure, use default. Common OpenAI- and Claude-style model names are routed to the matching tier automatically (see Model name routing). Anything unrecognised falls back to default.
4. You can send text and images (see Vision / images), OpenAI-style tools (see Tool calling) and response_format for JSON (see JSON / structured output). Don't send audio or files.
5. On a 503 or 429, retry with backoff (see Errors).

Endpoints

Supported request fields: model, messages (roles system, developer, user, assistant (including tool_calls), tool; string content, or arrays of text and image_url parts), stream, stream_options.include_usage, tools, tool_choice, parallel_tool_calls, response_format. Other fields such as temperature and max_tokens are accepted but ignored.

Models

Floating names (default, smart, fast) always point to the newest model of that tier. Versioned names are pinned.

The equivalents are only a rough guide, so any agent or developer can pick a tier quickly. The OpenAI column is an approximate capability match, not a benchmark claim.

Mapping rule: when asked for a Claude model, use its neutral name. Opus → smart*, Sonnet → default*, Haiku → fast*. Use the version suffix if it's listed, otherwise use the floating name. When asked for an OpenAI model, use the tier from the OpenAI column. The API always reports the neutral name in the model field.

Model name routing

You can also send a third-party model name directly. The gateway maps it to a tier, and the response's model field shows the neutral name it used. The first matching rule wins:

Examples

curl

curl https://llm.hisavage.com/v1/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"default","messages":[{"role":"system","content":"Be concise."},{"role":"user","content":"Hello!"}]}'

Streaming (Server-Sent Events, ends with data: [DONE]):

curl -N https://llm.hisavage.com/v1/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"fast","stream":true,"messages":[{"role":"user","content":"Count to 5"}]}'

Python (pip install openai)

import os
from openai import OpenAI

client = OpenAI(base_url="https://llm.hisavage.com/v1",
                api_key=os.environ["GATEWAY_API_KEY"], timeout=330, max_retries=3)

r = client.chat.completions.create(model="default",
        messages=[{"role": "user", "content": "Write a haiku about autumn."}])
print(r.choices[0].message.content)

for chunk in client.chat.completions.create(model="fast", stream=True,
        messages=[{"role": "user", "content": "Count to 5"}]):
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Node (npm i openai)

import OpenAI from "openai";
const client = new OpenAI({ baseURL: "https://llm.hisavage.com/v1",
  apiKey: process.env.GATEWAY_API_KEY, timeout: 330_000, maxRetries: 3 });

const r = await client.chat.completions.create({ model: "default",
  messages: [{ role: "user", content: "Hello!" }] });
console.log(r.choices[0].message.content);

const stream = await client.chat.completions.create({ model: "fast", stream: true,
  messages: [{ role: "user", content: "Count to 5" }] });
for await (const c of stream) process.stdout.write(c.choices[0]?.delta?.content ?? "");

Vision / images

Every model (default, smart, fast and all pinned versions) accepts images, using the standard OpenAI chat format. Put a content array in a user message with text parts and image_url parts. image_url.url can be:

• a data URL: data:image/png;base64,iVBORw0..., or
• a public https:// (or http://) URL, which the gateway downloads server-side.

detail is accepted and ignored. Images are numbered in order across the conversation, so you can refer to them as "the first image", "image 2", and so on.

Image limits

• Formats: PNG, JPEG, GIF, WebP (detected from the file bytes).
• Up to 10 MB per image and 10 images per request. The whole request body can be up to 25 MB (base64 adds about 33%).
• Remote URLs must be publicly reachable. Private, loopback, link-local and internal addresses are refused, and so is this server itself. Downloads time out after 15 s and follow at most 3 redirects. URLs with credentials aren't allowed.
• Image errors return 400 (or 413 if the image is too large) with code: "invalid_image" and a clear message.

curl (data URL)

IMG=$(base64 -w0 photo.png)   # macOS: base64 -i photo.png
curl https://llm.hisavage.com/v1/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"default","messages":[{"role":"user","content":[
        {"type":"text","text":"What does this image show?"},
        {"type":"image_url","image_url":{"url":"data:image/png;base64,'"$IMG"'"}}]}]}'

curl (remote URL)

curl https://llm.hisavage.com/v1/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"fast","messages":[{"role":"user","content":[
        {"type":"text","text":"Read any text in this image."},
        {"type":"image_url","image_url":{"url":"https://example.com/receipt.jpg"}}]}]}'

Python

import base64, os
from openai import OpenAI
client = OpenAI(base_url="https://llm.hisavage.com/v1", api_key=os.environ["GATEWAY_API_KEY"], timeout=330)

b64 = base64.b64encode(open("screenshot.png", "rb").read()).decode()
r = client.chat.completions.create(model="default", messages=[{"role": "user", "content": [
    {"type": "text", "text": "Transcribe the text in this screenshot."},
    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
    {"type": "image_url", "image_url": {"url": "https://example.com/second.jpg"}},
]}])
print(r.choices[0].message.content)

Node

import fs from "node:fs";
import OpenAI from "openai";
const client = new OpenAI({ baseURL: "https://llm.hisavage.com/v1", apiKey: process.env.GATEWAY_API_KEY, timeout: 330_000 });

const b64 = fs.readFileSync("chart.png").toString("base64");
const stream = await client.chat.completions.create({ model: "smart", stream: true, messages: [{ role: "user", content: [
  { type: "text", text: "Summarise this chart." },
  { type: "image_url", image_url: { url: `data:image/png;base64,${b64}` } },
] }] });
for await (const c of stream) process.stdout.write(c.choices[0]?.delta?.content ?? "");

Streaming and non-streaming both work with images.

Tool calling

Standard OpenAI function calling works on every model: tools (type function with a JSON Schema parameters), tool_choice ("auto" (default), "none", "required", or {"type":"function","function":{"name":"..."}}) and parallel_tool_calls (default true). Responses use the normal OpenAI shape: message.tool_calls[] with id (call_...), type: "function", function.name and function.arguments as a JSON string, plus finish_reason: "tool_calls". Send results back as role: "tool" messages with the matching tool_call_id. The gateway never runs your functions; your code does.

Tested with the official openai Python SDK (including chat.completions.stream and parse), langchain-openai (bind_tools) and openai-agents (Agent + function_tool via OpenAIChatCompletionsModel).

Python: full round trip

import json, os
from openai import OpenAI
client = OpenAI(base_url="https://llm.hisavage.com/v1", api_key=os.environ["GATEWAY_API_KEY"], timeout=330)

tools = [{"type": "function", "function": {
    "name": "get_weather", "description": "Get current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"},
                   "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}},
                   "required": ["city"], "additionalProperties": False}}}]

messages = [{"role": "user", "content": "What's the weather in Paris and Tokyo?"}]
r = client.chat.completions.create(model="default", messages=messages, tools=tools)
msg = r.choices[0].message            # finish_reason == "tool_calls", two parallel calls
messages.append(msg.model_dump(exclude_none=True))
for call in msg.tool_calls:
    args = json.loads(call.function.arguments)
    messages.append({"role": "tool", "tool_call_id": call.id,
                     "content": json.dumps({"city": args["city"], "temp_c": 17})})
final = client.chat.completions.create(model="default", messages=messages, tools=tools)
print(final.choices[0].message.content)

Node

import OpenAI from "openai";
const client = new OpenAI({ baseURL: "https://llm.hisavage.com/v1", apiKey: process.env.GATEWAY_API_KEY, timeout: 330_000 });
const tools = [{ type: "function", function: { name: "get_weather", description: "Get weather for a city",
  parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"] } } }];
const r = await client.chat.completions.create({ model: "fast", tools, tool_choice: "required",
  messages: [{ role: "user", content: "Weather in Lagos?" }] });
console.log(r.choices[0].message.tool_calls);  // [{ id: "call_...", function: { name: "get_weather", arguments: "{\"city\":\"Lagos\"}" } }]

Streaming (stream: true) also works. Tool calls arrive as standard delta.tool_calls chunks (index, id, function.name, then function.arguments fragments), which the SDKs assemble for you.

JSON / structured output

response_format works on every model:

• {"type": "json_object"}: the reply content is a single JSON object.
• {"type": "json_schema", "json_schema": {"name": "...", "schema": {...}, "strict": true}}: the reply is JSON that matches your schema. It's validated, and code fences or stray prose are stripped.

from pydantic import BaseModel
class City(BaseModel):
    name: str
    country: str
    landmarks: list[str]

r = client.chat.completions.parse(model="default", response_format=City,
        messages=[{"role": "user", "content": "Describe Paris."}])
print(r.choices[0].message.parsed)      # City(name='Paris', country='France', landmarks=[...])

curl https://llm.hisavage.com/v1/chat/completions -H "Authorization: Bearer $GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"fast","response_format":{"type":"json_object"},"messages":[{"role":"user","content":"Give a JSON object with keys name and age."}]}'

Caveats (tool calling and JSON)

• Both are emulated by the gateway through prompting, not native model features. They're reliable in testing, but:
• Slightly slower: the reply is generated in full before it's returned. With stream: true the chunks arrive all at once at the end rather than token by token. If the output is invalid, the gateway retries once, which adds a few seconds.
• Arguments and JSON are validated against a practical subset of JSON Schema: type, properties, required, additionalProperties, items, enum, const, anyOf/oneOf, minimum/maximum, and local $ref. Keep schemas simple.
• If output is still invalid after the retry, you get 502 invalid_model_output. The exception is non-strict json_schema, where you get the best valid JSON produced. With tool_choice: "auto" you get the plain-text reply as content.
• Tool calls and tool results in the history are shown to the model as text. Very large tool results count against the context like any other text.

Limits

• Text and images. Images are supported on every model (see Vision / images). Audio and file parts are dropped. Tool calling and JSON output are supported but emulated (see their sections).
• Concurrency: 2 requests run at a time and up to 20 more wait in a queue. Beyond that you get 429.
• Timeout: 5 minutes per request (returns 504). Set your client timeout a little above that, for example 330 s.
• Request body: 25 MB maximum (including base64 images).
• Each request is stateless, so send the full conversation in messages every time.
• usage token counts are approximate.

Errors and retries

Errors use the OpenAI shape: {"error":{"message":"...","type":"...","code":"..."}}.

If a stream fails mid-response, an error object is sent as a data: event, followed by data: [DONE].

Quick self-test

export GATEWAY_API_KEY=...   # your key
B=https://llm.hisavage.com/v1
curl -s -o /dev/null -w "%{http_code}\n" $B/models                                   # expect 401
curl -s -H "Authorization: Bearer $GATEWAY_API_KEY" $B/models                        # expect model list
curl -s -H "Authorization: Bearer $GATEWAY_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"fast","messages":[{"role":"user","content":"Reply with: ok"}]}' $B/chat/completions

The last call should return choices[0].message.content of about "ok", with "model":"fast".