How assessments work
Applies to every record in the directory. A recorded assessment is not a guarantee that a skill is safe.
Four separate questions
- Overall score
- A letter, A, B, C or F, summarizing findings in every category. The only place a letter appears.
- Security severity
- Critical, High, Medium or Low, plus Informational, None reported and Unavailable. Derived from security findings.
- Framework checks
- Pass, Fail or Not assessed, for each named framework.
- Operation status
- Whether a scan finished: Queued, Assessing, Completed or Couldn't assess. A completed scan is not a safety verdict.
Asset types
What the five agentic asset types mean. Repository is a separate directory kind.
- Skill
- A packaged instruction set an agent loads to do a task: a SKILL.md plus optional scripts and references.
- Prompt
- A reusable instruction or template you hand to a model.
- MCP server
- A service that exposes tools or data to an agent over the Model Context Protocol.
- Agent
- A model-driven worker that you configure with a model, instructions, and tools, then run to do a task. It is not a harness or a runtime.
- Agentic app
- A multi-agent application: several agents orchestrated together, often deployed as a service.
Overall score
Overall score and Security severity are derived separately, so a skill can have a clean overall score and a Medium severity, or a poor score and no security findings. Both are shown side by side.
How the score is calculated
The highest threshold reached determines the scanner's letter: F for one or more critical findings or three or more high findings; otherwise C for one or more high findings or three or more medium findings; otherwise B for one or more medium findings or four or more low findings; otherwise A. Informational findings do not affect the score.
Each successful scanner is scored separately. The stored record score uses the worst successful scanner score; errored or skipped scanners do not contribute one. There is no D.
Security severity
The rule, state by state
- Only findings in the security category count. Findings about best practices, descriptions, scripts, evals and structure change the overall score, not Security severity.
- Only scanners whose run completed count. If none did, the state is Unavailable. The coverage (for example "from 1 of 2 scanners; 1 couldn't assess") is shown beside the severity, and None reported or Informational on partial coverage is marked incomplete.
- The state is the highest severity among those findings, and each record's own findings are always one click away.
- Records whose findings were not kept use the captured security finding counts. Those counts do not include notes, so Informational cannot be shown for them.
- Critical
- At least one security finding with Critical severity from a scanner that ran.
- High
- No Critical; at least one High.
- Medium
- No Critical or High; at least one Medium.
- Low
- No Critical, High or Medium; at least one Low.
- Informational
- Only notes were reported as security findings. Notes are information, not problems.
- None reported
- No security finding at all from the scanners that ran. This is not proof that nothing is wrong.
- Unavailable
- No scanner run completed, or no security results were stored for the record. This is not a Low and not a clean result.
Framework checks
Every record shows Not assessed: no record carries coverage evidence.
Which framework, and why nothing is assessed
The only framework in scope is OWASP Top 10 for LLM Applications (2025), the OWASP list for LLM applications, not the OWASP web application Top 10 and not the agentic list. Category coverage is not compliance certification.
The scanners do not yet record which OWASP category a finding belongs to, so no category can be passed or failed honestly. Source: OWASP GenAI Security Project.
Disputes
What happens to a dispute
In the product, a finding keeps counting toward the overall score until an admin decides.
| Status | What it means | What it does to the finding and overall score |
|---|---|---|
| Under review | An admin hasn't decided yet. You can withdraw it. | The finding keeps counting toward the overall score, which stays as it is. |
| Upheld | An admin agreed the scanner got it wrong. | The finding is marked as a false positive and stops counting. The overall score is worked out again. |
| Dismissed | An admin reviewed it and the finding stands. | The finding keeps counting. The overall score stays as it is. |
| Withdrawn | You took it back before a decision. | Nothing changes. You can file a new dispute later. |
Admins decide with two buttons, Accept dispute and Reject dispute. Accept marks it Upheld, reject marks it Dismissed.
Historical result
What the archived marks meant
The current product records five consensus states. Consensus, security severity, overall score and operation status are separate things.
- Archived Verified
- Vettd passed and at least two external scanners agreed.
- Archived Pass
- A conclusive Pass consensus was recorded.
- Archived Warn
- Conditional consensus: review the concerns before use.
- Archived Fail
- Flagged consensus: inspect the concerns before use.
- Archived No result captured
- No conclusive consensus. Findings may still exist; this is not a Pass.
Reference records cannot establish current consensus, including Verified.
Signals and provenance
Signal categories and limits
- Safety and Reliability & Quality contain finding-oriented evidence. The stored overall score counts findings across categories; it is not a safety score.
- Performance, Cost and Compatibility describe measurements or declared information when captured.
- Popularity can include repository stars, with their source and reading date. Stars describe repository popularity, not skill use or safety.
- Characteristics describe recorded facts. A missing row is different from a measured zero when the source preserves that distinction; some source counts may be stored as zero when the upstream value was absent.
- Every signal records which process produced it and when it was observed. Signals are not refreshed on a schedule: a record is as fresh as its last scan or import.
Limits
A recorded assessment describes one assessed version of a skill, identified by its content hash (for a skill in a folder, the folder's Git tree SHA, not a commit), and the checks that completed. Static analysis does not run the skill or prove that it is safe for your use.
What an assessment cannot establish
Coverage can be incomplete, a source can change after assessment, and an operation can fail. A completed scan means the operation finished; it is not a safety verdict. A failed rescan leaves earlier evidence available. Line numbers appear only where a scanner's own text named one.
- Vettd analyzes the submitted artifact and public repository data. It does not run the asset or observe how it behaves at runtime.
- Compatibility reports what an asset declares it needs. It does not yet detect conflicts between an asset and a particular environment.
- Framework labels are reference context, not automated end-to-end compliance certifications.
- A scan with no findings is inconclusive, not a pass.