🎯 Three-Dimensional Scoring System¶

The authoritative guide to CapiscIO's scoring system - Used by both CapiscIO CLI and Python SDK

Why Three Dimensions?¶

The Problem with Single Scores:

Imagine you evaluate an agent and get a score of 67/100. What does that tell you?

• Is it compliant with the protocol but untrusted?
• Is it trusted but often unavailable?
• Is it a security risk or just poorly configured?

You can't tell. And you can't make informed decisions.

Real-World Scenario¶

You're building a payment processing agent. You discover two potential partner agents:

• Agent A: Compliance: 95, Trust: 45, Availability: 90

• Translation: Perfectly follows the protocol, always up, but no signature verification, missing provider info. Don't use for payments.

• Agent B: Compliance: 80, Trust: 95, Availability: 80

• Translation: Minor protocol quirks, occasionally slow, but fully signed, verified provider, excellent security. Good for payments.

A single score would show them both as ~75. Three dimensions tell you the REAL story.

Overview¶

Every agent card validation returns three independent score dimensions:

Each dimension has its own detailed breakdown, rating enum, and independent scoring logic.

📄 Dimension 1: Spec Compliance (100 points)¶

What it measures: How well the agent card adheres to the A2A v0.3.0 specification.

Why it matters: Protocol compliance ensures interoperability. An agent with poor compliance won't work correctly with other agents, registries, or tooling.

Scoring Breakdown¶

The compliance score uses a penalty-based model starting at 100 points. Each violation deducts a fixed penalty:

Core Fields¶

Checks for presence of all required A2A fields:

Skills Quality¶

Evaluates the quality of the skills array:

Format Compliance¶

Validates proper formatting:

Data Quality¶

Checks for data integrity:

Rating Levels¶

Common Issues¶

• ❌ Missing required fields like protocolVersion or capabilities
• ❌ Skills without tags (reduces discoverability)
• ❌ Invalid MIME types in input/output modes
• ❌ Using HTTP instead of HTTPS for URLs
• ❌ Invalid semantic version strings

🔐 Dimension 2: Trust (100 points)¶

What it measures: How trustworthy and secure is this agent? Can users verify its authenticity?

Why it matters: Trust is critical for adoption. Without signatures, users can't verify your agent's identity.

Scoring Model¶

Trust scoring uses a discrete assignment model based on the agent card's signature state:

The signature state is the dominant factor. Additional agent card metadata (provider info, security schemes, documentation URLs) contributes to the validation report details but the trust score is primarily driven by signature validity.

What Affects Trust¶

• Cryptographic signatures — JWS-signed agent cards receive significantly higher scores
• Trusted issuers — Configuring a trusted issuer list and matching against it yields the highest score
• HTTPS-only endpoints — HTTP URLs are flagged as security risks
• Provider information — Organization and URL presence improves trust indicators

Rating Levels¶

Common Issues¶

• ❌ No cryptographic signatures (score drops to 20)
• ❌ Invalid signature (score drops to 0)
• ❌ Using HTTP instead of HTTPS (flagged in validation report)
• ❌ Missing provider information (noted in trust details)

🚀 Dimension 3: Availability (100 points)¶

What it measures: Is the agent operationally available and responding correctly?

Why it matters: An agent with perfect compliance and trust scores is useless if it's unreachable. Availability testing catches deployment issues before users encounter them.

Scoring Breakdown¶

The availability score uses a 50/30/20 weighting across three categories:

Primary Endpoint (50 points)¶

Can we reach your agent?

• Endpoint responds (30 points) - Returns a valid HTTP response
• Fast response (10 points) - Responds in under 3 seconds
• CORS configured (5 points) - Proper CORS headers for web access
• Valid TLS certificate (5 points) - HTTPS certificate is valid and trusted

Transport Support (30 points)¶

Are the declared transports working?

• Preferred transport works (20 points) - Primary protocol responds correctly
• Additional interfaces (10 points) - Alternative transports also functional

Response Quality (20 points)¶

How well-formed are the responses?

• Valid A2A structure (10 points) - Responses follow A2A message format
• Proper content-type (5 points) - Correct HTTP headers
• Error handling (5 points) - Graceful error responses

Rating Levels¶

Common Issues¶

• ❌ Endpoint timeouts (> 3 seconds)
• ❌ Connection refused or DNS errors
• ❌ Invalid TLS certificates (expired, self-signed)
• ❌ Missing or incorrect CORS headers
• ❌ Responses not following A2A message structure
• ❌ Declared transports returning errors

Production Readiness¶

The scoring system automatically determines if an agent is production ready based on these thresholds:

Agents meeting all thresholds receive a "✅ Production ready" recommendation.

When to Use Each Dimension¶

Compliance Matters Most¶

Use cases: - Protocol testing and validation tools - Compliance auditing and certification - Interoperability testing between vendors - Agent registry quality gates

Decision logic: Reject agents with compliance < 90

Trust Matters Most¶

Use cases: - Financial transactions (payments, invoicing) - Identity verification (KYC, authentication) - Sensitive data handling (PII, health records, legal) - High-value API calls

Decision logic: Require trust ≥ 80 for sensitive operations

Availability Matters Most¶

Use cases: - Real-time systems (chat, live updates, streaming) - Critical path operations (task dependencies, workflows) - High-volume APIs (data sync, batch processing) - Time-sensitive integrations

Decision logic: Only call agents with availability ≥ 90

Using Scores¶

Scores are available via the CLI (--json for structured output) and the Python SDK (validate_agent_card()).

• CLI validation reference →
• Python SDK reference →

FAQ¶

print(result.compliance.breakdown.core_fields.score)
print(result.trust.breakdown.signatures.score)

See Also¶

• CapiscIO CLI Scoring - Command-line usage and flags
• Python SDK Reference - Python API and patterns
• Validation Concepts - Understanding validation architecture

Version: 1.0
Last Updated: October 2025
Maintained by: CapiscIO Documentation Team