Azure OpenAI

Configure Azure OpenAI specification.

When using Azure OpenAI LLMs, you have the choice of using a built-in Graphlit model, which accrues credits for usage, or using your own Azure OpenAI deployment.

Graphlit currently supports built-in Azure OpenAI models via the model field, such as: GPT35_TURBO_16K, GPT4, and GPT4_TURBO_128K. These models will always use the latest version, as defined by OpenAI.

As new models are released by Azure OpenAI, Graphlit will add model enums in future releases.

See here for the latest supported Azure OpenAI model enums.

By assigning the model field to CUSTOM, you also will need to assign the key, endpoint and deploymentName to use your own Azure OpenAI developer account.

Create Built-In Model Specification

Mutation:

mutation CreateSpecification($specification: SpecificationInput!) {
  createSpecification(specification: $specification) {
    id
    name
    state
    type
    serviceType
  }
}

Variables:

{
  "specification": {
    "type": "COMPLETION",
    "serviceType": "AZURE_OPEN_AI",
    "azureOpenAI": {
      "model": "GPT35_TURBO_16K"
    },
    "name": "Azure OpenAI"
  }
}

Response:

{
  "type": "COMPLETION",
  "serviceType": "AZURE_OPEN_AI",
  "id": "4405adc5-0dba-494f-8b8b-3489ffee0f5f",
  "name": "Azure OpenAI",
  "state": "ENABLED"
}

Create Custom Model Specification

Mutation:

mutation CreateSpecification($specification: SpecificationInput!) {
  createSpecification(specification: $specification) {
    id
    name
    state
    type
    serviceType
  }
}

Variables:

{
  "specification": {
    "type": "COMPLETION",
    "serviceType": "AZURE_OPEN_AI",
    "azureOpenAI": {
      "key": "redacted",
      "endpoint": "redacted",
      "deploymentName": "my-gpt-35-turbo-deployment",
      "model": "CUSTOM"
    },
    "name": "Azure OpenAI Custom"
  }
}

Response:

{
  "type": "COMPLETION",
  "serviceType": "AZURE_OPEN_AI",
  "id": "75bb4552-3518-4015-ac25-982c13ad7d02",
  "name": "Azure OpenAI Custom",
  "state": "ENABLED"
}