Skip to main content

Getting Started

The Vector Inference Platform is available to all users of Vector's Bon Echo cluster. At the time of this document, this includes the AI Engineering team, the Industry and a select group of researchers. (We will be opening up the environment to all Vector researchers later in 2026.)

There are currently 3 models available to use:

Usage Instructions

The Inference Platform is only available on the Bon Echo cluster. Start by logging in to the cluster:

ssh v.vectorinstitute.ai

Create a new Python script called qwen3-omni-30b-test.py:

from openai import OpenAI

client = OpenAI(
    base_url="http://aieng01:4000/v1/",
    timeout=300.0,
    api_key="<paste-your-api-key-here>"
)

response = client.chat.completions.create(
    model="openai/Qwen3-Omni-30B-A3B-Instruct",
    messages=[{"role": "user", "content": "Who is better, Messi or Ronaldo?"}]
)

print(response)

Now run the script:

python3 ./qwen3-omni-30b-test.py

You should expect output like the following:

ChatCompletion(id='chatcmpl-619eb6f0-035d-4999-8001-b8b9fb5d4fae', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Whether Lionel Messi or Cristiano Ronaldo is "better" depends on personal preferences, as both are extraordinary footballers with remarkable accomplishments. Here’s a balanced comparison to help understand why each player has their own strong supporters:\n\n🏆 **Goals and Scores**:\n- **Messi**: Over 850 career goals (all competitions). Known for scoring in every position on the pitch.\n- **Ronaldo**: Over 875 career goals. Excels as a prolific striker with great heading and finish.\n\n📊 **Achievements**:\n- **Messi**: \n  - 8 Ballon d’Ors (record) \n  - 4 Champions League titles\n  - 10 Serie A titles (with Barcelona & PSG)\n  - Copa America 2021, FIFA World Cup 2022\n- **Ronaldo**: \n  - 5 Ballon d’Ors (record until Messi’s 2019 surge, then surpassed)\n  - 5 Champions League titles\n  - 5 Premier League titles, 3 La Liga titles, 3 Serie A titles\n  - Euro 2016, FIFA World Cup 2014 (not champions)\n\n🔄 **Style of Play**:\n- **Messi**: Renowned for dribbling, vision, creativity, and playmaking. A classic number 10.\n- **Ronaldo**: A complete striker with physical power, aerial ability, goalscoring instinct, and stamina.\n\n🏅 **Legacy and Versatility**:\n- Messi is often considered the most complete talent ever — praised for his ability to dominate games with minimal touches.\n- Ronaldo is admired for his relentless work ethic, consistency, longevity, and ability to perform on the biggest stages.\n\n🎯 **Personal Preference**:\n- You might love Messi if you value artistry, tactics, and intricate passing.\n- You might prefer Ronaldo if you admire goal output, physicality, leadership, and big-game mentality.\n\n📣 Conclusion:\nThere is no definitive answer — both are legends of the game. Messi may have the edge in pure footballing skill and impact in top competitions, but Ronaldo’s goal-scoring consistency and ability to adapt across leagues show what\'s possible through hard work.', refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=None))], created=1769622523, model='Qwen3-Omni-30B-A3B-Instruct', object='chat.completion', service_tier=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=497, prompt_tokens=16, total_tokens=513, completion_tokens_details=None, prompt_tokens_details=None))

User Access Management

User access and API keys are all managed via a LiteLLM proxy service. This allows us to give various users different levels of access to certain models, while including retstrictions on budget (for commercial models), rate limits, and other access controls.

At the time of this document, all user access and API keys are controlled by the Infrastructure group within AI Engineering Team. Please direct any requests to our Slack channel at #vector-inference-platform. We plan to later make this a self-serve platform, so that individual team leads can control their own user access and API keys without needing to go through the AI Engineering team.

Back to top