Groq

Configure Groq specification.

When using Groq LLMs, you have the choice of using a built-in Graphlit model, which accrues credits for usage, or using your own Groq API key.

Graphlit currently supports these built-in Groq models via the model field: MIXTRAL_8X7B_INSTRUCT, LLAMA_3_70B, LLAMA_3_7B, LLAMA_2_70B, and GEMMA_7B.

By assigning the model field to CUSTOM, you also will need to assign the key and modelName to use your own Groq developer account.

You can find the list of current Groq models here, and use the model name syntax such as mixtral-8x7b-32768.

Create Built-In Model Specification

Mutation:

mutation CreateSpecification($specification: SpecificationInput!) {
  createSpecification(specification: $specification) {
    id
    name
    state
    type
    serviceType
  }
}

Variables:

{
  "specification": {
    "type": "COMPLETION",
    "serviceType": "GROQ",
    "groq": {
      "model": "MIXTRAL_8X7B_INSTRUCT"
    },
    "name": "Groq"
  }
}

Response:

{
  "type": "COMPLETION",
  "serviceType": "GROQ",
  "id": "a87ce6cd-75f4-4391-8d00-33149e9afc86",
  "name": "Groq",
  "state": "ENABLED"
}

Create Custom Model Specification

Mutation:

mutation CreateSpecification($specification: SpecificationInput!) {
  createSpecification(specification: $specification) {
    id
    name
    state
    type
    serviceType
  }
}

Variables:

{
  "specification": {
    "type": "COMPLETION",
    "serviceType": "GROQ",
    "groq": {
      "key": "redacted",
      "modelName": "mixtral-8x7b-32768",
      "model": "CUSTOM"
    },
    "name": "Groq Custom"
  }
}

Response:

{
  "type": "COMPLETION",
  "serviceType": "GROQ",
  "id": "3c7d107c-cc9f-4c2f-8ac5-0b0b7ddd2ddf",
  "name": "Groq Custom",
  "state": "ENABLED"
}