prompt-linter;

Prompt linter

Paste a system prompt and get the exact lines that contradict each other, the lines too vague to act on, and the checks it misses. Free, open source, no account.

How it works:

  • —The prompt is split into sentences and bullets; a sentence that wraps across lines stays one segment, and headings, code blocks and HTML comments are left out.
  • —Every line is checked against every other line, plus a vagueness check per line and seven whole-prompt checks, all in one round trip to Jev, TypeSafe AI’s decision-only model (a few parallel calls for long prompts).
  • —Jev returns calibrated probabilities, never text, so nothing in the report is made-up prose — our code decides the thresholds and the grade.
  • —A real lint of a 20-line prompt takes about a second and costs a hundredth of a cent.

Using the web app

Paste your system prompt and press “Lint prompt” or ⌘↵.

Coloured bars appear on the flagged lines: rose means a conflict (two lines that cannot both be obeyed), amber means a possible conflict, blue means too vague. Hover a finding in the side panel to highlight both lines it points at.

The grade block shows the letter grade and score. If you edit the prompt after a lint, a notice reads “Prompt changed. Lint again to update.”

Nothing you paste is stored. Results are kept in server memory for 10 minutes to serve repeated requests for the same prompt, and are never written to disk.

API

Request

POST https://prompt-linter.com/api/lint
Content-Type: application/json

{"prompt": "You are a helpful assistant.\nAlways respond in under 50 words.\nGive detailed, step-by-step explanations."}

No API key required. CORS is open, so browsers can call it directly.

Response (trimmed — 3 of 14 segments, 2 of 5 conflicts shown)

{
  "version": 1,
  "grade": "F",
  "score": 0,
  "summary": "4 conflicts, 1 possible conflict, 2 vague lines, 2 checks failed.",
  "segments": [
    {
      "id": 1,
      "line": 3,
      "from": 31,
      "to": 82,
      "text": "You are a friendly support assistant for Acme Bank.",
      "kind": "text"
    },
    {
      "id": 3,
      "line": 6,
      "from": 95,
      "to": 128,
      "text": "Always respond in under 50 words.",
      "kind": "text"
    },
    {
      "id": 4,
      "line": 7,
      "from": 131,
      "to": 189,
      "text": "Give detailed, step-by-step explanations for every answer.",
      "kind": "text"
    },
    "// … 11 more segments"
  ],
  "conflicts": [
    {
      "a": 3,
      "b": 4,
      "probability": 1,
      "confidence": 1,
      "severity": "conflict"
    },
    {
      "a": 8,
      "b": 9,
      "probability": 1,
      "confidence": 1,
      "severity": "conflict"
    },
    "// … 3 more conflicts"
  ],
  "vague": [
    {
      "segment": 14,
      "probability": 0.9
    },
    "// … 1 more"
  ],
  "checks": [
    {
      "id": "role",
      "label": "Role",
      "hint": "Say who the assistant is or what role it plays.",
      "passed": true,
      "probability": 0.98
    },
    "// … 6 more checks"
  ],
  "clarity": {
    "score": 0.06,
    "label": "Confusing, instructions conflict",
    "confidence": 0.94
  },
  "meta": {
    "model": "jev-1.13.0",
    "questions": 44,
    "latencyMs": 1050,
    "inputTokens": 3138,
    "outputTokens": 1945,
    "costUsd": 0.00013179600000000002,
    "truncated": false,
    "cached": false
  }
}

Fields

grade / score
Letter grade (A–F) and 0–100 score, computed in code from the findings.
summary
One plain sentence, e.g. "4 conflicts, 2 vague lines, no examples."
segments
Each linted piece of the prompt. id is 0-based. line is 1-based; endLine is present only when a sentence wraps across lines. from/to are character offsets in the original prompt (end exclusive). kind is text or heading.
conflicts
Pairs of segment ids (a, b) that cannot both be obeyed. probability is the highest score Jev gave in either direction. confidence is Jev's certainty in the stronger direction. severity is conflict (≥ 0.60) or possible (≥ 0.45).
vague
Segments flagged as too vague to act on. segment is the segment id. probability is Jev's score.
checks
Seven whole-prompt checks (role, task, format, examples, fallback, audience, boundaries). Each has an id, label, hint explaining what passing means, a boolean passed, and a probability.
clarity
Overall clarity on a 0–3 scale (0 = confusing, 3 = clear and unambiguous). label is the nearest level's description. confidence is Jev's certainty.
meta
model: versioned Jev model id. questions: number of questions sent. latencyMs: round-trip time of the Jev call. inputTokens / outputTokens. costUsd: cost of the Jev call. truncated: true when the prompt exceeded 200 segments and only the first were linted. cached: true when the result came from the server cache.

Errors

All errors return {"error": {"code", "message"}}.

400
invalid_request / too_short / too_long
429
rate_limited — honour Retry-After and back off
502
upstream_error — Jev returned an error
503
upstream_overloaded — retry with backoff

Limits

  • 20 to 12,000 characters per prompt
  • First 200 segments linted (extra segments are skipped, meta.truncated is true)
  • 20 requests per minute per IP
  • 10-minute server cache keyed on the prompt

Examples

curl

curl -s -X POST https://prompt-linter.com/api/lint \
  -H "Content-Type: application/json" \
  -d '{"prompt": "You are a helpful assistant.\nAlways respond in under 50 words.\nGive detailed, step-by-step explanations."}' \
  | jq '{grade: .grade, conflicts: [.conflicts[] | {lines: [.a, .b], severity: .severity, probability: .probability}]}'

JavaScript

const res = await fetch("https://prompt-linter.com/api/lint", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ prompt: "..." }),
});
const result = await res.json();

if ("error" in result) {
  console.error(result.error.code, result.error.message);
} else {
  console.log("Grade:", result.grade);
  for (const c of result.conflicts) {
    const a = result.segments.find(s => s.id === c.a);
    const b = result.segments.find(s => s.id === c.b);
    console.log(`Line ${a.line} ↔ line ${b.line} (${c.severity}, ${c.probability})`);
  }
}

Python

import requests

res = requests.post(
    "https://prompt-linter.com/api/lint",
    json={"prompt": "..."},
)
result = res.json()

if "error" in result:
    print(result["error"]["code"], result["error"]["message"])
else:
    print("Grade:", result["grade"])
    seg = {s["id"]: s for s in result["segments"]}
    for c in result["conflicts"]:
        a, b = seg[c["a"]], seg[c["b"]]
        print(f"Line {a['line']} ↔ line {b['line']} ({c['severity']}, {c['probability']})")

Add it to your project

Paste the prompt below into your coding agent (Claude Code, Cursor, Codex) to add prompt linting to any project. It finds your system prompts, lints each one, and wires up a CI gate in a single shot.

Copy prompt

Add prompt linting to this project using the prompt-linter API (free, no key, open source: https://github.com/shakee93/prompt-linter).

Endpoint: POST https://prompt-linter.com/api/lint
Body: {"prompt": "<the system prompt as one string>"}
200 response: {
  grade: "A"|"B"|"C"|"D"|"F", score: 0-100, summary: string,
  segments: [{id, line, endLine?, from, to, text, kind: "text"|"heading"}],  // line numbers are 1-based; endLine only when a sentence wraps
  conflicts: [{a, b, probability, confidence, severity: "conflict"|"possible"}],  // a and b are segment ids
  vague: [{segment, probability}],
  checks: [{id, label, hint, passed, probability}],
  clarity: {score: 0-3, label, confidence},
  meta: {model, questions, latencyMs, inputTokens, outputTokens, costUsd, truncated, cached}
}
Errors: {"error": {"code", "message"}} with 400, 429 (honour Retry-After), 502, 503. Retry 429/503 with backoff.
Limits: 20-12,000 characters per prompt, 20 requests per minute per IP.

Task: find every system prompt in this repo, add a script that lints each one and prints the conflicts and vague lines with their line numbers and the grade, and make CI fail when any prompt scores below B. Keep the HTTP call in one small module so it can be swapped for a self-hosted instance.

A few ways to take it further:

  • —CI gate: fail the build when any prompt scores below B.
  • —Pre-commit hook: lint changed prompt files before they land.
  • —Eval step: lint a prompt before a change ships to catch regressions early.

What it checks

Conflicts

Every pair of text segments is checked against each other. If the merged probability is ≥ 0.60, the pair is a conflict (rose). If it is ≥ 0.45, it is a possible conflict (amber).

Vague lines

Each text segment is checked individually. If the vagueness probability is ≥ 0.60, the line is flagged (blue).

Whole-prompt checks

Role
Say who the assistant is or what role it plays.
Task
State concretely what the assistant should do.
Output format
Specify the shape or length of the answer.
Examples
Include at least one example of a good answer.
Fallback
Say what to do when the assistant cannot help or is unsure.
Audience
Say who the answers are for.
Boundaries
State what the assistant must not do.

Clarity

Jev scores the prompt’s overall clarity on a 0–3 scale across four levels:

  1. 0Confusing, instructions conflict
  2. 1Somewhat unclear
  3. 2Mostly clear
  4. 3Clear and unambiguous

Grade formula

score = 100 − 15 × conflicts − 6 × possible conflicts − 5 × vague lines − 7 × failed checks + clarity adjustment (−10 to +10)

For prompts longer than 20 sentences, the conflict and vague penalties are scaled by 20 ÷ sentences, so a long prompt is judged by how dense its problems are, not their raw count. Check and clarity adjustments are always unscaled.

A≥ 90
B≥ 75
C≥ 60
D≥ 40
Fbelow 40

Limits

  • —It flags — it does not explain or rewrite. Labels and hints come from our code, not from Jev.
  • —Probabilities near a threshold are uncertain. A 0.61 conflict and a 0.61 possible are close calls.
  • —It reads text only. It does not parse structured data or code inside prompts.

Self-host

The full source is on GitHub under the MIT licence. To run it locally or on your own infrastructure:

Clone and install

git clone https://github.com/shakee93/prompt-linter
cd prompt-linter
pnpm install

Configure

cp .env.example .env.local
# Set TYPESAFE_API_KEY — get one at https://console.typesafe.ai/settings/keys

Run

pnpm dev

To deploy to Vercel, push to your fork and import the repo. vercel.json pins functions to sfo1, which is close to Jev’s servers and cuts about 250 ms from every lint. Set TYPESAFE_API_KEY and NEXT_PUBLIC_SITE_URL in the Vercel environment settings.