Model Runtime

You can onboard models deployed on any of the following platforms 

HuggingFace

AWS Bedrock

AzureAI

Google Vertex AI

This page presents list of fields required for onboarding a model.

HuggingFace

Inference Endpoint URL

Model ID

Bearer Token

AWS Bedrock

Region

Model ID

Access Key

Secret Key 

Azure AI

Inference Endpoint URL

OpenAI API Version

Model ID

API Key

Google Vertex

Project

Endpoint ID

Model ID

JSON file