LlamaIndex

LlamaIndex with donkey.llamaindex.llm("…"), an OpenAILike. The adapter sets is_chat_model=True — the OpenAILike default of False hits /completions — so calls go to the proxy’s /chat/completions route. Sync and async calls send through the SDK’s HTTP clients, so the run id reaches the proxy and last_call is populated. OpenAILike re-raises the openai error unchanged, so classify() types refusals.

Both scripts need a live gateway; there is no offline LlamaIndex script.

Install

Follow the examples setup first, then:

python -m pip install -e "../donkey-development-kit/python[llm,llamaindex]"
set -a; source .env.local; set +a

01 — complete() and chat()

python "demos/human-made/llamaindex/01 - basic-gw.py"

donkey = Donkey.from_env()
llm = donkey.llamaindex.llm("gpt-4o")
 
print(llm.complete("Say hello in exactly three words.").text)
 
reply = llm.chat([ChatMessage(role="user", content="Say goodbye in exactly three words.")])
print(reply.message.content)
print("total tokens", reply.raw.usage.total_tokens)
print("last_call   ", donkey.last_call.status.value, donkey.last_call.surface)

You should see: two greetings, total tokens, and last_call observed None. A 404 … /completions means is_chat_model was overridden back to False.

02 — Typed refusals, live

python "demos/human-made/llamaindex/02 - typed-refusals-live.py"

Three cases, each with its own Donkey: a contact record for PIIDetected, a model that does not exist for UpstreamRequestError, and wrong credentials for AuthError.

try:
    donkey.llamaindex.llm(model).complete(prompt)
    print(name, "NO REFUSAL")
except openai.APIStatusError as err:
    error = classify(err.response)
    print(name, "->", type(error).__name__, getattr(error, "entities", None))

Needs: llm-pii-detection-policy with Email and action Reject for the first case. You should see: <case> -> <Type> <entities> per case, or <case> NO REFUSAL.

Learn more: LlamaIndex

Source: demos/human-made/llamaindex/