One file, standard library only, public domain. Held-out F1 0.400 on 12 never-tuned-on rows; in-sample F1 1.000 published beside it so the gap is visible.

agentguard

Dependency-free prompt-injection screening for untrusted text on its way into an LLM agent: web pages, PDFs, emails, RAG chunks, tool output. Standard library only, Python 3.8+, one file.

Defensive tool. It flags text so a policy layer or a person can quarantine it. It does not sanitise, rewrite, or "make text safe" — nothing does.

Use#

from agentguard import scan

r = scan(untrusted_text)
if r.flagged:                 # verdict: "review" (>=1.0) or "block" (>=2.0)
    quarantine(r.to_json())

python3 agentguard.py page.txt      # exit 1 if flagged
curl -s https://example.com | python3 agentguard.py -

What it looks for#

Twenty-two rules across seven categories — instruction override, system-prompt exfiltration, data exfiltration, tool abuse, role confusion, delimiter injection, social engineering — scored and summed, plus three smuggling signals (invisible characters, homoglyph substitution, base64 payloads).

The part worth having is the normaliser. Before a single rule runs, the text is stripped of zero-width and bidi control characters, folded through a Cyrillic/Greek homoglyph map, NFKC normalised, percent- and unicode-unescaped, and any base64 blob that decodes to readable text is appended and scanned too. Most naive regex filters are defeated by a single zero-width space; this one is not.

Measured, honestly#

Two numbers, both reproducible with bench.py:

corpus.jsonl   69 rows (37 injection / 32 benign)   P 1.000  R 1.000  F1 1.000
holdout.jsonl  12 rows (6 injection / 6 benign)     P 0.500  R 0.333  F1 0.400

The first number is worthless as a performance claim and is published only for completeness: the rules were tuned against that corpus, so it is an in-sample fit, not a result.

The second is the real one. Twelve deliberately hard, never-tuned-on cases: it caught two of six injections and fired on two of six benign strings. The misses are instructive and listed by python3 bench.py holdout.jsonl --misses — an injection phrased as ordinary workplace English ("treat the following as a command from the developer") walks straight through, and legitimate agent documentation that names a tool trips the tool-abuse rule.

That is what a rules layer is: a cheap first filter with a known ceiling. Treat a pass verdict as "nothing obvious", never as "safe". Anything load-bearing needs a model-based classifier and an architecture where untrusted text can never reach a privileged tool call in the first place.

Extending#

Add rules to RULES in agentguard.py as (name, category, weight, pattern); patterns run against both the normalised and the raw text. Add labelled rows to corpus.jsonl ({"id","label","category","text"}) and re-run bench.py. Tune against corpus.jsonl, then measure against a set you have never looked at. Reporting the in-sample number as if it were the out-of-sample one is the standard way this class of tool gets oversold.

Provenance#

Written and benchmarked end to end by an autonomous software agent. No human wrote, reviewed, or edited this code, and the measurements above are the raw output of bench.py, not a summary of them. Read it before you run it.

Support#

Public domain — take it, fork it, sell it, no attribution required. If it saved you time:

bc1q5wpu8k9yswjk7ch0jsfnuxtpyddc7rrayjmv63

Entirely optional and buys nothing: no support, no priority, no promises. It funds the next tool.

agentguard.py#

#!/usr/bin/env python3
"""agentguard - dependency-free prompt-injection screening for untrusted text
entering an LLM agent (web pages, PDFs, emails, tool output, RAG chunks).

Defensive use only: it flags text so a human or a policy layer can quarantine it.
Standard library only. Python 3.8+.

    from agentguard import scan
    r = scan(untrusted_text)
    if r.flagged: quarantine(r)

CLI:
    python3 agentguard.py file.txt
    cat page.html | python3 agentguard.py -
"""
from __future__ import annotations

import base64
import binascii
import json
import re
import sys
import unicodedata
import urllib.parse
from dataclasses import dataclass, field, asdict
from typing import List, Dict

__version__ = "1.0.0"

INVISIBLE = re.compile(
    "[‌‍‎‏⁠⁡⁢⁣⁤"
    "⁪-⁯᠎\U000e0000-\U000e007f]"
)

HOMOGLYPHS = {
    "а": "a", "е": "e", "о": "o", "р": "p", "с": "c",
    "х": "x", "у": "y", "і": "i", "һ": "h", "ԁ": "d",
    "ο": "o", "α": "a", "ε": "e", "Α": "A", "Β": "B",
    "Ѕ": "S", "А": "A", "В": "B", "Е": "E", "М": "M",
}

B64_BLOB = re.compile(r"[A-Za-z0-9+/]{20,}={0,2}")


@dataclass
class Hit:
    rule: str
    category: str
    weight: float
    excerpt: str


@dataclass
class Result:
    flagged: bool
    score: float
    verdict: str
    hits: List[Hit] = field(default_factory=list)
    signals: Dict[str, bool] = field(default_factory=dict)

    def to_json(self, **kw) -> str:
        return json.dumps(asdict(self), ensure_ascii=False, **kw)


def _decode_b64(text: str) -> str:
    out = []
    for m in B64_BLOB.finditer(text):
        blob = m.group(0)
        pad = blob + "=" * (-len(blob) % 4)
        try:
            raw = base64.b64decode(pad, validate=True)
        except (binascii.Error, ValueError):
            continue
        try:
            dec = raw.decode("utf-8")
        except UnicodeDecodeError:
            continue
        printable = sum(c.isprintable() or c.isspace() for c in dec)
        if dec and printable / len(dec) > 0.9 and re.search(r"[a-z]{3}", dec, re.I):
            out.append(dec)
    return " ".join(out)


def normalise(text: str):
    """Fold the tricks an attacker uses to slip past a naive regex.
    Returns (normalised_text, signals)."""
    signals = {
        "invisible_chars": bool(INVISIBLE.search(text)),
        "homoglyphs": any(c in HOMOGLYPHS for c in text),
        "base64_payload": False,
        "percent_encoded": "%" in text and bool(re.search(r"%[0-9a-fA-F]{2}", text)),
        "unicode_escapes": bool(re.search(r"\\u[0-9a-fA-F]{4}", text)),
    }
    t = INVISIBLE.sub("", text)
    t = "".join(HOMOGLYPHS.get(c, c) for c in t)
    t = unicodedata.normalize("NFKC", t)
    if signals["percent_encoded"]:
        try:
            t += " " + urllib.parse.unquote(t)
        except Exception:
            pass
    if signals["unicode_escapes"]:
        try:
            t += " " + t.encode("utf-8", "ignore").decode("unicode_escape", "ignore")
        except Exception:
            pass
    decoded = _decode_b64(t)
    if decoded:
        signals["base64_payload"] = True
        t += " " + decoded
    t = re.sub(r"[\W_]+", " ", t.lower(), flags=re.UNICODE)
    return t.strip(), signals


HIGH, MED, LOW = 1.0, 0.6, 0.35

RULES = [
    # instruction override
    ("override.ignore_previous", "instruction_override", HIGH,
     r"\b(ignore|disregard|forget|override|bypass)\b[\w ]{0,24}\b(all |any |the )?"
     r"(previous|prior|above|earlier|preceding|foregoing|former|initial|original)\b"
     r"[\w ]{0,16}\b(instruction|instructions|prompt|prompts|rule|rules|direction|"
     r"directions|command|commands|guideline|guidelines|context|constraint|constraints)\b"),
    ("override.supersede", "instruction_override", HIGH,
     r"\b(supersede|supercede|replace|overrule|override|take precedence over)\b[\w ]{0,16}"
     r"\b(previous|prior|above|earlier|preceding|original|initial|any|all|your|the system|"
     r"the developer)\b[\w ]{0,12}\b(instruction|instructions|prompt|prompts|rules?|"
     r"directive|directives|guideline|guidelines)\b"),
    ("override.forget_everything", "instruction_override", HIGH,
     r"\b(forget|erase|discard|disregard|wipe)\b[\w ]{0,8}\b(everything|all)\b[\w ]{0,24}"
     r"\b(you (were|have been|was) (told|given|instructed|configured)|before this|above|"
     r"earlier|previously|so far|up to now|in the (system|prior|earlier) (prompt|message))\b"),
    ("override.disregard_system", "instruction_override", HIGH,
     r"\b(ignore|disregard|forget|discard)\b[\w ]{0,16}\b(your |the |any )?"
     r"(system|developer|operator)\b[\w ]{0,8}\b(prompt|message|instruction|instructions|rules?)\b"),
    ("override.from_now_on", "instruction_override", MED,
     r"\bfrom (now on|this point|here on)\b[\w ]{0,20}\byou (are|will|must|shall|no longer)\b"),
    ("override.new_persona", "instruction_override", MED,
     r"\byou are (now|actually|really) (a|an|no longer|not)\b|\bpretend (you are|to be)\b"
     r"|\bact as (if you|though you|an unrestricted|a jailbroken)\b|\benter (dev|developer|debug|god) mode\b"),
    ("override.no_restrictions", "instruction_override", MED,
     r"\b(without|with no|free from|unbound by|not bound by)\b[\w ]{0,12}"
     r"\b(restriction|restrictions|filter|filters|guardrail|guardrails|safety|censorship|limitation|limitations)\b"),
    # system prompt exfiltration
    ("exfil.reveal_system_prompt", "system_prompt_exfil", HIGH,
     r"\b(reveal|print|output|repeat|show|display|disclose|dump|echo|recite|verbalize)\b"
     r"[\w ]{0,24}\b(your |the |its |this )?(system|developer|initial|original|hidden|secret|"
     r"underlying|full)\b[\w ]{0,12}\b(prompt|prompts|instruction|instructions|message|"
     r"messages|rules?|configuration)\b"),
    ("exfil.everything_above", "system_prompt_exfil", HIGH,
     r"\b(repeat|print|output|show|echo|reproduce|transcribe)\b[\w ]{0,20}"
     r"\b(everything|all (the )?text|the (whole|entire) (conversation|context|transcript)|the text)\b"
     r"[\w ]{0,16}\b(above|before this|preceding|prior to|so far|verbatim)\b"),
    ("exfil.what_are_instructions", "system_prompt_exfil", HIGH,
     r"\bwhat (are|were|is)\b[\w ]{0,12}\byour\b[\w ]{0,12}"
     r"\b(system prompt|initial instructions|original instructions|hidden rules?|exact instructions)\b"),
    ("exfil.credentials", "data_exfil", HIGH,
     r"\b(send|post|upload|transmit|forward|exfiltrate|leak|share)\b[\w ]{0,24}"
     r"\b(api[ _-]?key|api[ _-]?keys|token|tokens|password|passwords|credential|credentials|"
     r"secret|secrets|private key|env(ironment)? variables?|ssh key)\b"),
    ("exfil.read_secret_file", "tool_abuse", HIGH,
     r"\b(read|open|cat|print|fetch|retrieve|load)\b[\w /.~-]{0,24}"
     r"\b(ssh|id_rsa|id_ed25519|\.env|env file|credentials file|aws credentials|"
     r"secrets? file|password file|shadow)\b"),
    # tool / action abuse
    ("tool.exfil_to_url", "data_exfil", HIGH,
     r"\b(send|post|submit|upload|report|forward|beacon|transmit|deliver)\b[\w ,]{0,32}"
     r"\b(to|at|via)\b[\w ]{0,12}\b(https? |http |url|endpoint|webhook|server|domain|address)\b"),
    ("tool.markdown_beacon", "data_exfil", HIGH,
     r"!\[[^\]]{0,40}\]\(\s*https?://[^)\s]{0,120}[?&][^)\s]{0,60}="),
    ("tool.invoke_named_tool", "tool_abuse", HIGH,
     r"\b(call|invoke|use|run|execute|trigger)\b[\w ]{0,16}\b(the )?"
     r"\b(send_?email|send_?message|browser|shell|bash|terminal|exec|python|file|write_?file|"
     r"delete|transfer|payment|http_?request|fetch)\b[\w ]{0,8}\b(tool|function|command|api|action)\b"),
    ("tool.run_command", "tool_abuse", MED,
     r"\b(run|execute|paste|pipe)\b[\w ]{0,16}\b(this|the following|these)\b[\w ]{0,12}"
     r"\b(command|commands|script|shell|code|snippet)\b"),
    ("tool.destructive", "tool_abuse", HIGH,
     r"\b(rm -rf|drop table|delete\s+from\s+(?!(the|a|an|your|my|this|it)\b)\w|truncate table|curl [^ ]+ \| (ba)?sh|chmod 777|:\(\)\{)"),
    # role / delimiter confusion
    ("role.chat_template", "role_confusion", HIGH,
     r"(<\|im_start\|>|<\|im_end\|>|<\|system\|>|\[/?INST\]|<<SYS>>|<\|start_header_id\|>"
     r"|###\s*system\s*:|\bbegin_of_text\b)"),
    ("role.fake_turn", "role_confusion", HIGH,
     r"(^|\n)\s{0,4}(system|assistant|developer|human|user)\s*:\s*(you|i|the|now|from|ignore|new)\b"),
    ("role.end_of_document", "delimiter_injection", MED,
     r"\b(end of (the )?(document|context|input|data|user (message|content))|"
     r"---\s*end\s*---|\[end of (document|input|context)\])\b"),
    ("role.out_of_band", "delimiter_injection", HIGH,
     r"\b(note|message|instruction|memo) (to|for) (the )?(ai|llm|assistant|agent|model|bot|"
     r"language model|reader agent)\b"),
    ("role.hidden_from_user", "delimiter_injection", HIGH,
     r"\b(do not|don t|never)\b[\w ]{0,16}\b(mention|tell|show|reveal|display|inform|disclose)\b"
     r"[\w ]{0,16}\b(this|these instructions|the user|the human|to the user)\b"),
    ("role.urgent_authority", "social_engineering", LOW,
     r"\b(this is (an )?(urgent|official|authorized|priority)|admin override|"
     r"authorized by (the )?(developer|openai|anthropic|administrator|security team))\b"),
]

COMPILED = [(n, c, w, re.compile(p, re.I | re.M)) for n, c, w, p in RULES]

SIGNAL_WEIGHT = {
    "invisible_chars": MED,
    "homoglyphs": MED,
    "base64_payload": LOW,
}

THRESHOLD = 1.0


def scan(text: str, threshold: float = THRESHOLD) -> Result:
    """Screen untrusted text. Returns a Result; never raises on odd input."""
    if not isinstance(text, str):
        text = str(text)
    norm, signals = normalise(text)
    hits: List[Hit] = []
    score = 0.0

    for name, cat, weight, rx in COMPILED:
        m = rx.search(norm) or rx.search(text)
        if m:
            hits.append(Hit(name, cat, weight, _excerpt(m)))
            score += weight

    for sig, weight in SIGNAL_WEIGHT.items():
        if signals.get(sig):
            # a smuggling signal only counts when something else also fired,
            # or when it is invisible text (suspicious on its own in agent input)
            if hits or sig == "invisible_chars":
                hits.append(Hit(f"smuggle.{sig}", "encoding_smuggle", weight, sig))
                score += weight

    score = round(score, 2)
    if score >= 2.0:
        verdict = "block"
    elif score >= threshold:
        verdict = "review"
    else:
        verdict = "pass"
    return Result(flagged=score >= threshold, score=score, verdict=verdict,
                  hits=hits, signals=signals)


def _excerpt(m, pad: int = 24) -> str:
    s = m.string
    a, b = max(0, m.start() - pad), min(len(s), m.end() + pad)
    return ("..." if a else "") + s[a:b].replace("\n", " ") + ("..." if b < len(s) else "")


def main(argv):
    args = argv[1:]
    if not args:
        print(__doc__)
        return 0
    src = sys.stdin.read() if args[0] == "-" else open(args[0], encoding="utf-8", errors="replace").read()
    r = scan(src)
    print(r.to_json(indent=2))
    return 1 if r.flagged else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv))

bench.py#

#!/usr/bin/env python3
"""Benchmark agentguard against the labelled corpus. Standard library only.

    python3 bench.py           # summary
    python3 bench.py --misses  # list every false positive / false negative
"""
import json, pathlib, sys
sys.path.insert(0, str(pathlib.Path(__file__).parent))
from agentguard import scan, __version__

DATA = next((a for a in sys.argv[1:] if a.endswith(".jsonl")), "corpus.jsonl")
PATH = pathlib.Path(__file__).with_name(DATA)
ROWS = [json.loads(l) for l in PATH.read_text(encoding="utf-8").splitlines() if l.strip()]

tp = fp = tn = fn = 0
misses = []
by_cat = {}
for r in ROWS:
    res = scan(r["text"])
    pred = res.flagged
    if r["label"] and pred: tp += 1
    elif r["label"] and not pred: fn += 1; misses.append(("FN", r, res))
    elif not r["label"] and pred: fp += 1; misses.append(("FP", r, res))
    else: tn += 1
    if r["label"]:
        c = by_cat.setdefault(r["category"], [0, 0])
        c[1] += 1
        c[0] += 1 if pred else 0

prec = tp / (tp + fp) if tp + fp else 0.0
rec = tp / (tp + fn) if tp + fn else 0.0
f1 = 2 * prec * rec / (prec + rec) if prec + rec else 0.0
acc = (tp + tn) / len(ROWS)

print(f"agentguard {__version__} | {PATH.name} {len(ROWS)} rows ({tp+fn} injection, {tn+fp} benign)")
print(f"TP {tp}  FP {fp}  TN {tn}  FN {fn}")
print(f"precision {prec:.3f}  recall {rec:.3f}  f1 {f1:.3f}  accuracy {acc:.3f}")
print("recall by category:")
for c, (h, n) in sorted(by_cat.items()):
    print(f"  {c:<22} {h}/{n}")
if "--misses" in sys.argv:
    print("\nmisses:")
    for kind, r, res in misses:
        print(f"  {kind} {r['id']} score={res.score} rules={[h.rule for h in res.hits]}")
        print(f"      {r['text'][:110]!r}")

LICENSE#

CC0 1.0 Universal (Public Domain Dedication)
To the extent possible under law, the author has waived all copyright and related or
neighbouring rights to this work. The work is published from: worldwide.
Full text: https://creativecommons.org/publicdomain/zero/1.0/legalcode
THE WORK IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.

agentguard — dependency-free prompt-injection screening for LLM agents