Skip to main content

LiteLLM

LiteLLM supports three ways to integrate custom models:

Add the provider to litellm/llms/openai_like/providers.json:

{
  "hpc_ai": {
    "base_url": "https://api.hpc-ai.com/inference/v1",
    "api_key_env": "HPC_AI_API_KEY"
  }
}

Or configure the environment variable and use:

export HPC_AI_API_KEY="your-hpc-ai-api-key"
import litellm
response = litellm.completion(
    model='hpc_ai/minimax/minimax-m2.5',
    messages=[{'role': 'user', 'content': 'test'}]
)

Method 2: LiteLLM Library Directly​

pip install litellm
import litellm
import os

os.environ["HPC_AI_API_KEY"] = "your-hpc-ai-api-key"

response = litellm.completion(
    model='hpc_ai/minimax/minimax-m2.5',
    messages=[{'role': 'user', 'content': 'Hello'}]
)
print(response.choices[0].message.content)

Method 3: LiteLLM Proxy​

  1. Create config.yaml:
model_list:
  - model_name: minimax/minimax-m2.5
    litellm_params:
      model: minimax/minimax-m2.5
      api_base: https://api.hpc-ai.com/inference/v1
      api_key: os.environ/HPC_AI_API_KEY
  1. Set environment variable and start Proxy:
export HPC_AI_API_KEY="your-hpc-ai-api-key"
pip install 'litellm[proxy]'
litellm --config config.yaml
  1. Call via unified API:
curl https://your-proxy:4000/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -d '{"model": "minimax/minimax-m2.5", "messages": [{"role": "user", "content": "Hello"}]}'