New Model Onboarding
You can onboard LLMs from your environment and launch scans to detect vulnerabilities associated with the LLMs.
To create a new model:
- Navigate to Inventory > Models tab and click New Model.
- Enter the values in the Basic Details page.
- Configure Scan Settings for the model scan.
- Enter comments to be associated with the model.
- Confirm the settings and click Create Model.
Once the model is created, it is available in the Models list. You can view, edit, delete the model, or launch a scan.
Steps to Onboard a New Model
Basic Details
Provide the model name and add a tag to be associated with the model.
From the Model Runtime list, select the platform, where the model is deployed and provide the required details:
Enter the following information
- Select a region where the model is deployed
- Model ID
- a unique access key and corresponding secret key for authentication
Enter the following information:
- Inference Endpoint URL in the specified format
- OpenAI API version
- Model ID
- API Key for authentication
Enter the following information:
- Project name
- Endpoint ID
- Model ID
- Authentication Details - You can enter authentication details in one of the following ways:
- Enter the contents in JSON format in the field
{
"type": "service_account",
"project_id": "your-project-id",
"private_key_id": "your-private-key-id",
"private_key": "—BEGIN PRIVATE KEY—Your Private Key—END PRIVATE KEY-—",
"client_email": "service-account@project-id.iam.gserviceaccount.com",
"client id": "123456789",
"auth_uri”: "https://accounts.google.eom/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_urr': "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/service-account%40project-id.iam.gserviceaccount.com"
}
- Upload a JSON file containing authentication details. You can browse and select the file to upload or drag and drop the authentication JSON file.
Enter the following information:
- Inference Endpoint URL in the specified format
- Model ID
- Bearer token for authentication.
You must add the bearer token in the correct format. If you do not have a bearer token, do not enter any value in the Bearer Token field.
Scan Settings
In the Scan Settings, select an option profile and scanner appliance to be used for the model scan.
Option Profile
The option profile contains the scan settings and detection scope for the model scan. You can select from the existing option profiles or create a new option profile using the Create Record option.
For details on new option profile creation, see Option Profile Creation.
Scanner Appliance
Select one of the following options:
- Select External for perimeter scanning.
- Select Internal and select the scanner appliance from the list.
- Select Tags and select a tag. The scanner appliance associated with the selected tag is selected for model scan.
Comments
Enter comments to be associated with the model that is created.
Confirmation
Review the settings defined for the model and then click Create Model.