Skip to main content

Getting Started

The Vector Inference Platform is available to all usersVector Institute community members. For an up-to-date list of Vector'savailable models and their specifications, visit Bon Echo clusterinference.vectorinstitute.ai. At

Prerequisites

Install the timeOpenAI ofPython thisclient:

pip document,install this includes the AI Engineering team, the Industry and a select group of researchers. (We will be opening up the environment to all Vector researchers later in 2026.)


There are currently 2 models available to use:


openai

Usage Instructions

The Inferenceplatform Platform is available viaexposes an OpenAI-compatible endAPI managedat https://proxy.vectorinstitute.ai/v1. You can use it as a drop-in replacement for any OpenAI client by changing the AIbase_url Engineeringand team.model Start by creatingparameters.

Create a new Python script called qwen3-omni-30b-test.py:

from openai import OpenAI

client = OpenAI(
    base_url="https://proxy.vectorinstitute.ai/v1",
    timeout=300.0,
    api_key="<paste-your-api-key-herekey> (see below about getting api keys)"
)

responsestream = client.chat.completions.create(
    model="hosted_vllm/Qwen3-Omni-30B-A3B-Instruct"<model-id>",  # see inference.vectorinstitute.ai for available models
    messages=[{"role": "user", "content": "WhoExplain isattention better,mechanisms Messiin transformers."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or Ronaldo?"", end="", flush=True)

You can also use curl:

curl https://proxy.vectorinstitute.ai/v1/chat/completions \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id>",
    "max_tokens"messages": 128}[{"role": "user", "content": "Hello!"}]
  )

print(response)

}'

Getting an API Key

Now run the script:

python3 ./qwen3-omni-30b-test.py

You should expect output like the following:

ChatCompletion(id='chatcmpl-619eb6f0-035d-4999-8001-b8b9fb5d4fae', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Whether Lionel Messi or Cristiano Ronaldo is "better" depends on personal preferences, as both are extraordinary footballers with remarkable accomplishments. Here’s a balanced comparison to help understand why each player has their own strong supporters:\n\n🏆 **Goals and Scores**:\n- **Messi**: Over 850 career goals (all competitions). Known for scoring in every position on the pitch.\n- **Ronaldo**: Over 875 career goals. Excels as a prolific striker with great heading and finish.\n\n📊 **Achievements**:\n- **Messi**: \n  - 8 Ballon d’Ors (record) \n  - 4 Champions League titles\n  - 10 Serie A titles (with Barcelona & PSG)\n  - Copa America 2021, FIFA World Cup 2022\n- **Ronaldo**: \n  - 5 Ballon d’Ors (record until Messi’s 2019 surge, then surpassed)\n  - 5 Champions League titles\n  - 5 Premier League titles, 3 La Liga titles, 3 Serie A titles\n  - Euro 2016, FIFA World Cup 2014 (not champions)\n\n🔄 **Style of Play**:\n- **Messi**: Renowned for dribbling, vision, creativity, and playmaking. A classic number 10.\n- **Ronaldo**: A complete striker with physical power, aerial ability, goalscoring instinct, and stamina.\n\n🏅 **Legacy and Versatility**:\n- Messi is often considered the most complete talent ever — praised for his ability to dominate games with minimal touches.\n- Ronaldo is admired for his relentless work ethic, consistency, longevity, and ability to perform on the biggest stages.\n\n🎯 **Personal Preference**:\n- You might love Messi if you value artistry, tactics, and intricate passing.\n- You might prefer Ronaldo if you admire goal output, physicality, leadership, and big-game mentality.\n\n📣 Conclusion:\nThere is no definitive answer — both are legends of the game. Messi may have the edge in pure footballing skill and impact in top competitions, but Ronaldo’s goal-scoring consistency and ability to adapt across leagues show what\'s possible through hard work.', refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=None))], created=1769622523, model='Qwen3-Omni-30B-A3B-Instruct', object='chat.completion', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=497, prompt_tokens=16, total_tokens=513, completion_tokens_details=None, prompt_tokens_details=None))

User Access Management

All user access and API keys are controlledmanaged by the Infrastructure group within the AI Engineering Team.team. PleaseTo directrequest anyaccess, requestsreach toout ourvia the Slack channel at #vector-inference-platform.

Listing

APIAvailable KeysModels

You can retrieve the current list of enabled models programmatically via the API:

curl https://proxy.vectorinstitute.ai/v1/models \
  -H "Authorization: Bearer <your-api-key>"

We'llOr usesimply CyberArkvisit to distribute API Keys. Project leads are responsibleinference.vectorinstitute.ai for sharing these keys with their teams using a "Securedvisual Note" item.overview.

Back to top