Skip to content
AlicanTorun
Let's talk

The platform is live.

Take a look
The Lab

The method · 40 checks · 8 principles

What I actually check

The Trust Audit says what a system should do. This is the method underneath it: the questions I put to a real codebase, what clears each one, and the shape that fails.

A rubric is easy to agree with and impossible to act on. So here is the whole method, in the open. Eight principles, each one broken into the concrete questions I ask of your repository, what a passing answer looks like, and the anti-pattern I am looking for. Nothing here is generated. It is the list I work from.

Each check names where to look. Most are answerable by reading a handful of files, which is the point: a check you cannot verify is an opinion. Run them yourself, or point your own agent at them over MCP and have it report back against your codebase.

01 · 4 checks

Own your context

Own your context, rent the model.

  • Can a client take everything they own in one call?

    High
    Clears it
    One export path that enumerates every tenant-scoped table, with a test that derives the table list from the schema rather than restating it.
    Fails it
    An export covering the tables someone remembered, which silently stops covering the ones added later.
    Where to look
    the export routeany export testcolumns named tenant_id / workspace_id / account_id

  • Do the vectors live in a database you control?

    High
    Clears it
    Embeddings stored in your own store, re-derivable from source documents you also hold.
    Fails it
    The only copy of the index lives inside a vendor, so leaving means re-ingesting everything from scratch, if you still have it.
    Where to look
    the vector store clientthe ingestion pathwhether the source text is retained after embedding

  • Is the model behind one seam, or wired into call sites?

    Medium
    Clears it
    A single provider module and model ids as typed constants, so swapping a model is one edit.
    Fails it
    A vendor SDK imported directly wherever generation happens, with model ids as loose strings.
    Where to look
    imports of provider SDKs across the treewhere model id strings appear

  • Does delete reach the things that are not rows?

    Critical
    Clears it
    One teardown primitive that destroys external resources before the database cascade, reused by every delete path.
    Fails it
    A foreign-key cascade treated as the whole deletion contract, while provisioned projects, repositories, subscriptions and VMs keep running.
    Where to look
    delete servicesany column holding an external id or refwhat happens to those on delete

02 · 5 checks

Grounded or it doesn't answer

If it can't cite it, it shouldn't say it.

  • Does the answering call see only what was retrieved?

    Critical
    Clears it
    Retrieval runs first and its output is the only source material in the prompt.
    Fails it
    A system prompt full of background the model can blend with, so you cannot tell a retrieved fact from a remembered one.
    Where to look
    the prompt assembly for the answer pathwhat else is concatenated alongside the retrieved chunks

  • Are citations built from retrieved ids, or written by the model?

    Critical
    Clears it
    The renderer resolves citations against the chunks actually retrieved, and a citation that resolves to nothing is caught.
    Fails it
    The model emits citation markers as free text and the UI renders them, so a fabricated source looks identical to a real one.
    Where to look
    how citations are parsed and renderedany check that a cited id was in the retrieved set

  • Is refusal enforced in code, or only requested in the prompt?

    Critical
    Clears it
    A predicate decides refusal from the retrieval result, independent of what the model chose to say.
    Fails it
    The instruction says to refuse when unsure, and nothing verifies that it did.
    Where to look
    a grounding or refusal predicatewhether the answer path can return an ungrounded answer at all

  • Does the interface claim a refusal only when one happened?

    High
    Clears it
    What the user is told is derived from the same signal that is recorded, and the showable claim is never weaker than the recorded one.
    Fails it
    A reassuring card saying the system stopped rather than guess, rendered over an answer it actually generated.
    Where to look
    the refusal or trust componentthe field it readswhether that field is the recorded one

  • Does the eval set contain questions that must NOT be answered?

    High
    Clears it
    A golden set with explicit refusal cases, run when retrieval, chunking, the model or the grounding prompt changes.
    Fails it
    An eval that only measures recall, which a model that answers everything confidently passes.
    Where to look
    the eval setthe ratio of answerable to unanswerable cases

03 · 5 checks

Untrusted content is data, not instructions

Retrieved and user content is data the model reads, never orders it follows.

  • Is retrieved and user content marked as untrusted where it enters the prompt?

    High
    Clears it
    Every untrusted block is delimited and labelled as reference material that is never an instruction.
    Fails it
    Document text pasted into the prompt indistinguishable from your own instructions.
    Where to look
    how retrieved chunks are formatted into the promptany untrusted marker

  • Can retrieved content reach a tool call?

    Critical
    Clears it
    The call that answers from untrusted context has no tools bound to it, so there is nothing for injected text to trigger.
    Fails it
    One agent loop with both retrieval output and write tools available, where a crafted document can call them.
    Where to look
    the tool list on the answer callwhether retrieval and tool use share a loop

  • Does the component that reads hostile input have tools at all?

    Critical
    Clears it
    Anything parsing inbound email, uploads or third-party content returns structured data and can call nothing.
    Fails it
    An extractor with tool access, so the content it parses decides what it does.
    Where to look
    inbound email, upload and webhook processingthe tool set on each

  • Are tool arguments validated server-side before they execute?

    High
    Clears it
    Every tool argument is schema-checked at the boundary, with bounds shared with the API schema rather than restated.
    Fails it
    Arguments trusted because the model produced them, so a prompted id reaches a query.
    Where to look
    tool definitionswhere their inputs are parsedwhether bounds are duplicated

  • If the agent runs commands, is the set allowed or merely filtered?

    Critical
    Clears it
    An explicit allowlist of permitted commands, denying everything else by default.
    Fails it
    A denylist of dangerous patterns, which is a guess about every command you did not think of.
    Where to look
    any command execution pathhow the permitted set is expressed

04 · 5 checks

The agent proposes, a human decides

Autonomy stops at anything you can't undo.

  • Does agent-authored code reach the default branch directly?

    Critical
    Clears it
    Changes land on a branch as a pull request a human reviews. The default branch is never written by the agent.
    Fails it
    Commits pushed straight to main, where the review step is whatever the reviewer notices afterwards.
    Where to look
    the push pathwhich ref it targetswhether a PR is opened

  • Is approval required before anything you cannot undo?

    Critical
    Clears it
    Irreversible actions block on explicit human approval. Reversible ones do not, so the gate keeps its meaning.
    Fails it
    Either no gate at all, or a gate on everything, which trains people to click through it.
    Where to look
    the approval flowwhich actions require itwhich do not

  • Is the diff that was approved provably the diff that lands?

    High
    Clears it
    The base is compared between approval and push, and an ambiguous or truncated comparison blocks rather than proceeds.
    Fails it
    Approval on a diff computed against a base that has since moved, so the merge contains work nobody reviewed.
    Where to look
    what happens between approval and pushany base comparisonwhat it does when it cannot tell

  • Is the record of who did what append-only and attributed?

    High
    Clears it
    An audit trail nothing updates or deletes, recording actor, action and time, including the operator's own access.
    Fails it
    A log with an update path, or one that records the client's actions but not the vendor's.
    Where to look
    the audit tableits grantswhether operator access is recorded too

  • Is agent-written code scanned for secrets before it is pushed?

    High
    Clears it
    A scan runs before the push, and a hit blocks it rather than annotating it.
    Fails it
    Scanning after the fact, or not at all, so a generated example key becomes a real commit.
    Where to look
    any secret scanwhether it gates the push or reports on it

05 · 5 checks

Two layers or it's one bug from a breach

Auth in the app is not isolation. Enforce it at the database too.

  • Does every content row carry a tenant id, and does every query filter on it?

    Critical
    Clears it
    One tenant column on every content table, and services that filter by the RESOLVED tenant on every read and write.
    Fails it
    A table that joins its way to a tenant, so one missing join is a cross-tenant read.
    Where to look
    the schema for tenant columnsservice queries for the filterany table lacking one

  • Is there a second layer under the application checks?

    Critical
    Clears it
    Row-level security on tenant tables with policies, and closed grants on anything holding tokens, money or infrastructure state.
    Fails it
    Application checks only, so one missing filter is a breach rather than a bug.
    Where to look
    which tables have RLS enabledwhich have policieswhich grants exist for the anonymous and authenticated roles

  • Where does the tenant id come from?

    Critical
    Clears it
    Resolved from the authenticated user on every request. A tenant id in a request body is never trusted.
    Fails it
    An id read from the body or a query parameter, which makes every endpoint a tenant switcher.
    Where to look
    route handlerswhat the tenant id is derived fromany body field naming a tenant

  • Does a foreign id return 404 or 403?

    Medium
    Clears it
    404, so the response does not confirm that the resource exists.
    Fails it
    403 on a resource belonging to someone else, which is an existence oracle you can enumerate.
    Where to look
    the not-found path in the tenant gatewhat status a foreign id produces

  • Can a service-role or admin key reach the browser?

    Critical
    Clears it
    Privileged clients are constructed only in server modules, and the build would fail if one were imported into client code.
    Fails it
    A single shared client, or a key in a variable the bundler exposes.
    Where to look
    where the privileged client is builtwhether those modules are server-onlypublic environment variable names

06 · 5 checks

One boundary: API-first, validated, typed

One server boundary that validates everything. No business logic in the client.

  • Is every input schema-validated at one server boundary?

    High
    Clears it
    One validation helper and schemas that live with the domain, imported by every surface that accepts that input.
    Fails it
    Hand-rolled checks per handler, and bounds restated in each tool and form, which drift apart.
    Where to look
    the validation helperwhere schemas are definedany handler parsing input itself

  • Does authorization happen before any data work?

    Critical
    Clears it
    The tenant gate is the first statement in the handler, returning early on failure.
    Fails it
    Validation, lookups or writes before the gate, so an unauthorized request still moves the database.
    Where to look
    the first lines of each handleranything before the gate

  • Can the browser write to the database directly?

    Critical
    Clears it
    All mutations go through the server boundary. The client holds no write credential.
    Fails it
    A public key with insert rights, where correctness depends on rules written somewhere else.
    Where to look
    client-side database callsthe grants on the public role

  • Do handlers state their return type?

    Medium
    Clears it
    An explicit return annotation on every handler, so a drift in what it returns is a compile error.
    Fails it
    Inferred return types, which the compiler accepts happily while the response shape changes underneath.
    Where to look
    handler signaturesany without an annotation

  • Do route bodies do the work?

    Medium
    Clears it
    Handlers gate, validate and shape the response. Services own the work and the tenant scoping.
    Fails it
    Database work inside handlers, or services returning HTTP responses, which puts the security boundary in two places.
    Where to look
    handler lengthdatabase calls inside handlersHTTP types imported into services

07 · 6 checks

Bounded by design

An agent with no limits is a bill and an outage waiting to happen.

  • What stops an agent loop that does not converge?

    High
    Clears it
    Explicit step, retry and cost ceilings, enforced by the runtime rather than by the prompt.
    Fails it
    A loop that runs until the model decides to stop, which is a bill with no upper bound.
    Where to look
    the agent loopmax step and retry settingsany per-run cost ceiling

  • Does every outbound call carry a timeout?

    Medium
    Clears it
    A timeout on every external call, chosen deliberately rather than inherited from a default.
    Fails it
    A hung provider holding a request open until the platform kills it, taking the work with it.
    Where to look
    fetch and SDK calls to third partiestimeout or abort signals

  • Is a retried unit of work charged and applied once?

    Critical
    Clears it
    A deterministic key derived from the CONTENT of the work, so a resumed or re-delivered job collapses onto the same record.
    Fails it
    A key derived from position, a cursor or a parent id, which produces a fresh key on resume and bills the same work twice.
    Where to look
    idempotency keyswhat each is derived fromwhether a resumed batch would produce the same one

  • What happens at zero balance, and what happens when the balance cannot be read?

    Critical
    Clears it
    A hard, explicit block at zero, and a read failure that fails closed rather than assuming credit.
    Fails it
    Silent degradation, or a database error read as no balance found and treated as fine.
    Where to look
    the spend gateevery balance readwhat each does with an error result

  • What recovers work that dies between two states?

    High
    Clears it
    Every non-terminal status has a sweeper that does not depend on the in-request error handler, because a hard kill never runs it.
    Fails it
    A status moved to terminal only inside a catch block, so an out-of-memory kill leaves it stuck forever.
    Where to look
    every non-terminal status valuewhat sets it to terminalwhether anything sweeps it

  • Are unbounded reads capped, and does the surface admit when it truncated?

    Medium
    Clears it
    A cap on any query that grows with the data, and a surface that states it is showing a subset.
    Fails it
    A page that loads everything, or worse, one that caps silently and renders the capped number as the total.
    Where to look
    queries with no limitoperator pageswhether a truncated count is labelled

08 · 5 checks

You can tell when it's wrong

If you can't trace a bad answer back to its cause, you can't trust the good ones.

  • Is there a catch block that discards its error?

    High
    Clears it
    Every failure path logs with enough context to identify the tenant and the operation.
    Fails it
    An empty catch, or one that swallows and returns a default, which is a failure nobody will ever see.
    Where to look
    catch blockswhich ones logwhich return a fallback quietly

  • Does the data layer throw on failure, or resolve with an error you have to read?

    Critical
    Clears it
    If it resolves, every write reads the error and handles it, and webhook paths rethrow so the provider re-delivers.
    Fails it
    An awaited write whose result is discarded, which is a fully silent data loss on a constraint or a timeout.
    Where to look
    awaited writes with no destructured errorwebhook handlerswhether a failure there returns success

  • What does a failed read on a billing path return?

    Critical
    Clears it
    It throws. A transient database error is never allowed to read as zero, empty or not found on a money path.
    Fails it
    A wallet or subscription read that swallows its error, so a blip becomes a wrong charge or a free tier granted forever.
    Where to look
    reads on wallet, subscription, ledger and customer recordswhat each does with an error

  • Where do the loud signals actually go?

    High
    Clears it
    Logs reach a drain someone can query after the fact, so a mitigation whose enforcement is that it logs loudly is real.
    Fails it
    Structured error events written to standard output with no drain, which is detection nobody receives.
    Where to look
    the logger implementationwhether a drain or sink is configured

  • Can you get from a wrong answer back to what produced it?

    High
    Clears it
    A record per generation carrying what was retrieved, which model ran, what it cost, and whether it was grounded.
    Fails it
    A response with no trace, so the only debugging tool is asking again and hoping it happens twice.
    Where to look
    the per-generation recordwhether retrieval context is recoverable from it