Mankomail

Évaluation de prompt ​

Les routes sont relatives à <PUBLIC_BASE_URL> ; les corps de requête et de réponse sont en JSON sauf mention contraire. L’authentification, les portées, la pagination et le format des erreurs sont décrits dans les guides de l’API REST.

POST /api/v1/workflows/{id}/nodes/{nodeId}/eval ​

Lancer un run d’évaluation sur un nœud de catégorisation

Fait tourner le nœud du brouillon ENREGISTRÉ (un ai.categorize) sur un échantillon de mails et le note contre le jeu de référence. 202 : les cas arrivent par le WebSocket, le détail se lit par GET /eval-runs/{id}. Un nœud pas encore enregistré est eval.draft_not_saved.

Accès — Session de membre ou clé d’API portant workflows:write.

Paramètres

NomOùTypeRequis
idcheminstringoui
nodeIdcheminstringoui

Corps de la requête (application/json)

ChampTypeRequis
sampleobjectoui
Schéma JSON
json
{
  "type": "object",
  "properties": {
    "sample": {
      "oneOf": [
        {
          "type": "object",
          "properties": {
            "kind": { "type": "string", "const": "recent" },
            "limit": { "default": 50, "type": "integer", "minimum": 10, "maximum": 200 }
          },
          "required": [ "kind" ]
        },
        {
          "type": "object",
          "properties": { "kind": { "type": "string", "const": "reference" } },
          "required": [ "kind" ]
        }
      ]
    }
  },
  "required": [ "sample" ]
}

Réponses

202 — Le run, planifié.

ChampTypeRequis
runobjectoui
Schéma JSON
json
{
  "type": "object",
  "properties": {
    "run": {
      "type": "object",
      "properties": {
        "id": { "type": "string" },
        "workflowId": { "type": "string" },
        "nodeId": { "type": "string" },
        "status": {
          "type": "string",
          "enum": [ "queued", "running", "succeeded", "failed" ]
        },
        "prompt": { "type": "string" },
        "categories": { "type": "array", "items": { "type": "string" } },
        "multiLabel": { "type": "boolean" },
        "model": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
        "totals": {
          "type": "object",
          "properties": {
            "total": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
            "done": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
            "failed": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }
          },
          "required": [ "total", "done", "failed" ],
          "additionalProperties": false
        },
        "createdAt": { "type": "string" },
        "finishedAt": { "anyOf": [ { "type": "string" }, { "type": "null" } ] }
      },
      "required": [
        "id",
        "workflowId",
        "nodeId",
        "status",
        "prompt",
        "categories",
        "multiLabel",
        "model",
        "totals",
        "createdAt",
        "finishedAt"
      ],
      "additionalProperties": false
    }
  },
  "required": [ "run" ],
  "additionalProperties": false
}

400 — La requête ne respecte pas son schéma.

401 — Aucune session ni clé d’API valide (auth.unauthenticated).

403 — Refusé : rôle insuffisant (auth.forbidden), portée manquante (api_key.scope_missing, details.required la nomme) ou route fermée aux clés (api_key.session_required).

429 — La clé d’API dépasse son débit (api_key.rate_limited) ; Retry-After dit quand réessayer.

Cette opération accepte un en-tête Idempotency-Key : rejouer la même requête avec la même clé rend la réponse d’origine au lieu d’agir deux fois. Voir Idempotence.

Codes d’erreur — request.bad_request, workflow.not_found, eval.draft_not_saved, eval.node_not_categorizer, eval.node_params_invalid, eval.sample_empty, eval.llm_unavailable. Les codes communs (request.bad_request, auth.unauthenticated, api_key.scope_missing, api_key.rate_limited…) s’appliquent à toutes les routes ; voir Erreurs.

GET /api/v1/workflows/{id}/nodes/{nodeId}/eval-runs ​

Lister les runs d’évaluation d’un nœud

Chaque run porte le prompt et les catégories avec lesquels il a tourné : deux runs se comparent côte à côte.

Accès — Session de membre ou clé d’API portant workflows:read.

Paramètres

NomOùTypeRequis
idcheminstringoui
nodeIdcheminstringoui

Réponses

200 — Les runs.

ChampTypeRequis
runsobject[]oui
Schéma JSON
json
{
  "type": "object",
  "properties": {
    "runs": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": { "type": "string" },
          "workflowId": { "type": "string" },
          "nodeId": { "type": "string" },
          "status": {
            "type": "string",
            "enum": [ "queued", "running", "succeeded", "failed" ]
          },
          "prompt": { "type": "string" },
          "categories": { "type": "array", "items": { "type": "string" } },
          "multiLabel": { "type": "boolean" },
          "model": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
          "totals": {
            "type": "object",
            "properties": {
              "total": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
              "done": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
              "failed": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }
            },
            "required": [ "total", "done", "failed" ],
            "additionalProperties": false
          },
          "createdAt": { "type": "string" },
          "finishedAt": { "anyOf": [ { "type": "string" }, { "type": "null" } ] }
        },
        "required": [
          "id",
          "workflowId",
          "nodeId",
          "status",
          "prompt",
          "categories",
          "multiLabel",
          "model",
          "totals",
          "createdAt",
          "finishedAt"
        ],
        "additionalProperties": false
      }
    }
  },
  "required": [ "runs" ],
  "additionalProperties": false
}

401 — Aucune session ni clé d’API valide (auth.unauthenticated).

403 — Refusé : rôle insuffisant (auth.forbidden), portée manquante (api_key.scope_missing, details.required la nomme) ou route fermée aux clés (api_key.session_required).

429 — La clé d’API dépasse son débit (api_key.rate_limited) ; Retry-After dit quand réessayer.

Codes d’erreur — workflow.not_found. Les codes communs (request.bad_request, auth.unauthenticated, api_key.scope_missing, api_key.rate_limited…) s’appliquent à toutes les routes ; voir Erreurs.

GET /api/v1/workflows/{id}/nodes/{nodeId}/few-shot ​

Obtenir des exemples few-shot tirés de mes corrections

Des exemples équilibrés tirés du jeu de référence, prêts à coller dans un prompt. Toujours 200 : un jeu vide rend un bloc vide.

Accès — Session de membre ou clé d’API portant workflows:read.

Paramètres

NomOùTypeRequis
idcheminstringoui
nodeIdcheminstringoui
limitrequêteintegernon

Réponses

200 — Les exemples.

ChampTypeRequis
blockstringoui
examplesobject[]oui
messageIdsstring[]oui
referenceCountintegeroui
Schéma JSON
json
{
  "type": "object",
  "properties": {
    "block": { "type": "string" },
    "examples": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "messageId": { "type": "string" },
          "subject": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
          "from": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
          "excerpt": { "type": "string" },
          "categories": { "type": "array", "items": { "type": "string" } }
        },
        "required": [ "messageId", "subject", "from", "excerpt", "categories" ],
        "additionalProperties": false
      }
    },
    "messageIds": { "type": "array", "items": { "type": "string" } },
    "referenceCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }
  },
  "required": [ "block", "examples", "messageIds", "referenceCount" ],
  "additionalProperties": false
}

400 — La requête ne respecte pas son schéma.

401 — Aucune session ni clé d’API valide (auth.unauthenticated).

403 — Refusé : rôle insuffisant (auth.forbidden), portée manquante (api_key.scope_missing, details.required la nomme) ou route fermée aux clés (api_key.session_required).

429 — La clé d’API dépasse son débit (api_key.rate_limited) ; Retry-After dit quand réessayer.

Codes d’erreur — request.bad_request, workflow.not_found. Les codes communs (request.bad_request, auth.unauthenticated, api_key.scope_missing, api_key.rate_limited…) s’appliquent à toutes les routes ; voir Erreurs.

GET /api/v1/eval-runs/{id} ​

Lire un run d’évaluation

Les cas, paginés par curseur, et le score, calculé sur TOUT le run. Le run d’un autre membre est un 404.

Accès — Session de membre ou clé d’API portant workflows:read.

Paramètres

NomOùTypeRequis
idcheminstringoui
cursorrequêtestringnon
limitrequêteintegernon

Réponses

200 — Le run, son score et une page de cas.

ChampTypeRequis
runobjectoui
scoreobjectoui
casesobject[]oui
nextCursorstring | nulloui
Schéma JSON
json
{
  "type": "object",
  "properties": {
    "run": {
      "type": "object",
      "properties": {
        "id": { "type": "string" },
        "workflowId": { "type": "string" },
        "nodeId": { "type": "string" },
        "status": {
          "type": "string",
          "enum": [ "queued", "running", "succeeded", "failed" ]
        },
        "prompt": { "type": "string" },
        "categories": { "type": "array", "items": { "type": "string" } },
        "multiLabel": { "type": "boolean" },
        "model": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
        "totals": {
          "type": "object",
          "properties": {
            "total": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
            "done": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
            "failed": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }
          },
          "required": [ "total", "done", "failed" ],
          "additionalProperties": false
        },
        "createdAt": { "type": "string" },
        "finishedAt": { "anyOf": [ { "type": "string" }, { "type": "null" } ] }
      },
      "required": [
        "id",
        "workflowId",
        "nodeId",
        "status",
        "prompt",
        "categories",
        "multiLabel",
        "model",
        "totals",
        "createdAt",
        "finishedAt"
      ],
      "additionalProperties": false
    },
    "score": {
      "type": "object",
      "properties": {
        "scored": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
        "exactMatches": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
        "accuracy": {
          "anyOf": [
            { "type": "number", "minimum": 0, "maximum": 1 },
            { "type": "null" }
          ]
        },
        "multiLabel": { "type": "boolean" },
        "perCategory": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "category": { "type": "string" },
              "expectedCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
              "predictedCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
              "truePositives": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
              "falsePositives": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
              "falseNegatives": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 },
              "precision": {
                "anyOf": [
                  { "type": "number", "minimum": 0, "maximum": 1 },
                  { "type": "null" }
                ]
              },
              "recall": {
                "anyOf": [
                  { "type": "number", "minimum": 0, "maximum": 1 },
                  { "type": "null" }
                ]
              },
              "f1": {
                "anyOf": [
                  { "type": "number", "minimum": 0, "maximum": 1 },
                  { "type": "null" }
                ]
              }
            },
            "required": [
              "category",
              "expectedCount",
              "predictedCount",
              "truePositives",
              "falsePositives",
              "falseNegatives",
              "precision",
              "recall",
              "f1"
            ],
            "additionalProperties": false
          }
        },
        "confusion": {
          "oneOf": [
            {
              "type": "object",
              "properties": {
                "kind": { "type": "string", "const": "single" },
                "labels": { "type": "array", "items": { "type": "string" } },
                "counts": {
                  "type": "array",
                  "items": {
                    "type": "array",
                    "items": {
                      "type": "integer",
                      "minimum": 0,
                      "maximum": 9007199254740991
                    }
                  }
                }
              },
              "required": [ "kind", "labels", "counts" ],
              "additionalProperties": false
            },
            {
              "type": "object",
              "properties": {
                "kind": { "type": "string", "const": "multi" },
                "perCategory": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "properties": {
                      "category": { "type": "string" },
                      "truePositives": {
                        "type": "integer",
                        "minimum": 0,
                        "maximum": 9007199254740991
                      },
                      "falsePositives": {
                        "type": "integer",
                        "minimum": 0,
                        "maximum": 9007199254740991
                      },
                      "falseNegatives": {
                        "type": "integer",
                        "minimum": 0,
                        "maximum": 9007199254740991
                      },
                      "trueNegatives": {
                        "type": "integer",
                        "minimum": 0,
                        "maximum": 9007199254740991
                      }
                    },
                    "required": [
                      "category",
                      "truePositives",
                      "falsePositives",
                      "falseNegatives",
                      "trueNegatives"
                    ],
                    "additionalProperties": false
                  }
                }
              },
              "required": [ "kind", "perCategory" ],
              "additionalProperties": false
            }
          ]
        }
      },
      "required": [
        "scored",
        "exactMatches",
        "accuracy",
        "multiLabel",
        "perCategory",
        "confusion"
      ],
      "additionalProperties": false
    },
    "cases": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": { "type": "string" },
          "mailboxId": { "type": "string" },
          "from": {
            "anyOf": [
              {
                "type": "object",
                "properties": { "name": { "type": "string" }, "email": { "type": "string" } },
                "required": [ "email" ],
                "additionalProperties": false
              },
              { "type": "null" }
            ]
          },
          "subject": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
          "snippet": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
          "receivedAt": { "type": "string" },
          "folderLabels": { "type": "array", "items": { "type": "string" } },
          "flags": {
            "type": "object",
            "properties": {
              "seen": { "type": "boolean" },
              "flagged": { "type": "boolean" },
              "draft": { "type": "boolean" },
              "sent": { "type": "boolean" }
            },
            "required": [ "seen", "flagged", "draft", "sent" ],
            "additionalProperties": false
          },
          "signals": {
            "type": "object",
            "properties": {
              "isAutoReply": { "type": "boolean" },
              "isNoReply": { "type": "boolean" },
              "isMailingList": { "type": "boolean" },
              "isFromSelf": { "type": "boolean" }
            },
            "required": [ "isAutoReply", "isNoReply", "isMailingList", "isFromSelf" ],
            "additionalProperties": false
          },
          "hasAttachments": { "type": "boolean" },
          "status": { "type": "string", "enum": [ "pending", "succeeded", "failed" ] },
          "predicted": {
            "anyOf": [
              {
                "type": "object",
                "propertyNames": { "type": "string", "minLength": 1 },
                "additionalProperties": { "type": "boolean" }
              },
              { "type": "null" }
            ]
          },
          "expected": {
            "anyOf": [
              {
                "type": "object",
                "propertyNames": { "type": "string", "minLength": 1 },
                "additionalProperties": { "type": "boolean" }
              },
              { "type": "null" }
            ]
          },
          "error": { "anyOf": [ { "type": "string" }, { "type": "null" } ] }
        },
        "required": [
          "id",
          "mailboxId",
          "from",
          "subject",
          "snippet",
          "receivedAt",
          "folderLabels",
          "flags",
          "signals",
          "hasAttachments",
          "status",
          "predicted",
          "expected",
          "error"
        ],
        "additionalProperties": false
      }
    },
    "nextCursor": { "anyOf": [ { "type": "string" }, { "type": "null" } ] }
  },
  "required": [ "run", "score", "cases", "nextCursor" ],
  "additionalProperties": false
}

400 — La requête ne respecte pas son schéma.

401 — Aucune session ni clé d’API valide (auth.unauthenticated).

403 — Refusé : rôle insuffisant (auth.forbidden), portée manquante (api_key.scope_missing, details.required la nomme) ou route fermée aux clés (api_key.session_required).

429 — La clé d’API dépasse son débit (api_key.rate_limited) ; Retry-After dit quand réessayer.

Codes d’erreur — request.bad_request, eval.run_not_found. Les codes communs (request.bad_request, auth.unauthenticated, api_key.scope_missing, api_key.rate_limited…) s’appliquent à toutes les routes ; voir Erreurs.

PUT /api/v1/eval-runs/{id}/cases/{messageId}/expected ​

Corriger les étiquettes attendues d’un cas

Un remplacement des étiquettes attendues, écrit dans le cas ET dans le jeu de référence en une transaction. Une étiquette inconnue du run est eval.unknown_category.

Accès — Session de membre ou clé d’API portant workflows:write.

Paramètres

NomOùTypeRequis
idcheminstringoui
messageIdcheminstringoui

Corps de la requête (application/json)

ChampTypeRequis
expectedobjectoui
Schéma JSON
json
{
  "type": "object",
  "properties": {
    "expected": {
      "type": "object",
      "propertyNames": { "type": "string", "minLength": 1 },
      "additionalProperties": { "type": "boolean" }
    }
  },
  "required": [ "expected" ]
}

Réponses

200 — Le cas corrigé et la nouvelle taille du jeu de référence.

ChampTypeRequis
caseobjectoui
referenceCountintegeroui
Schéma JSON
json
{
  "type": "object",
  "properties": {
    "case": {
      "type": "object",
      "properties": {
        "id": { "type": "string" },
        "mailboxId": { "type": "string" },
        "from": {
          "anyOf": [
            {
              "type": "object",
              "properties": { "name": { "type": "string" }, "email": { "type": "string" } },
              "required": [ "email" ],
              "additionalProperties": false
            },
            { "type": "null" }
          ]
        },
        "subject": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
        "snippet": { "anyOf": [ { "type": "string" }, { "type": "null" } ] },
        "receivedAt": { "type": "string" },
        "folderLabels": { "type": "array", "items": { "type": "string" } },
        "flags": {
          "type": "object",
          "properties": {
            "seen": { "type": "boolean" },
            "flagged": { "type": "boolean" },
            "draft": { "type": "boolean" },
            "sent": { "type": "boolean" }
          },
          "required": [ "seen", "flagged", "draft", "sent" ],
          "additionalProperties": false
        },
        "signals": {
          "type": "object",
          "properties": {
            "isAutoReply": { "type": "boolean" },
            "isNoReply": { "type": "boolean" },
            "isMailingList": { "type": "boolean" },
            "isFromSelf": { "type": "boolean" }
          },
          "required": [ "isAutoReply", "isNoReply", "isMailingList", "isFromSelf" ],
          "additionalProperties": false
        },
        "hasAttachments": { "type": "boolean" },
        "status": { "type": "string", "enum": [ "pending", "succeeded", "failed" ] },
        "predicted": {
          "anyOf": [
            {
              "type": "object",
              "propertyNames": { "type": "string", "minLength": 1 },
              "additionalProperties": { "type": "boolean" }
            },
            { "type": "null" }
          ]
        },
        "expected": {
          "anyOf": [
            {
              "type": "object",
              "propertyNames": { "type": "string", "minLength": 1 },
              "additionalProperties": { "type": "boolean" }
            },
            { "type": "null" }
          ]
        },
        "error": { "anyOf": [ { "type": "string" }, { "type": "null" } ] }
      },
      "required": [
        "id",
        "mailboxId",
        "from",
        "subject",
        "snippet",
        "receivedAt",
        "folderLabels",
        "flags",
        "signals",
        "hasAttachments",
        "status",
        "predicted",
        "expected",
        "error"
      ],
      "additionalProperties": false
    },
    "referenceCount": { "type": "integer", "minimum": 0, "maximum": 9007199254740991 }
  },
  "required": [ "case", "referenceCount" ],
  "additionalProperties": false
}

400 — La requête ne respecte pas son schéma.

401 — Aucune session ni clé d’API valide (auth.unauthenticated).

403 — Refusé : rôle insuffisant (auth.forbidden), portée manquante (api_key.scope_missing, details.required la nomme) ou route fermée aux clés (api_key.session_required).

429 — La clé d’API dépasse son débit (api_key.rate_limited) ; Retry-After dit quand réessayer.

Codes d’erreur — request.bad_request, eval.run_not_found, eval.case_not_found, eval.unknown_category. Les codes communs (request.bad_request, auth.unauthenticated, api_key.scope_missing, api_key.rate_limited…) s’appliquent à toutes les routes ; voir Erreurs.