Step 4: Understanding Reports¶

CapiscIO uses a three-dimensional scoring system to give you a complete picture of your agent's health. Let's understand what each score means.

The Three Dimensions¶

graph TD
    A[Agent Card] --> B[Validation Engine]
    B --> C[Compliance Score]
    B --> D[Trust Score]
    B --> E[Availability Score]
    C --> F[Overall Assessment]
    D --> F
    E --> F

Compliance Score¶

The Compliance Score measures how well your agent card follows the A2A protocol specification.

What's Checked¶

Improving Your Score¶

Our sample agent card scores 75/100. Let's improve it:

{
  "name": "My First Agent",
  "description": "A sample A2A agent for learning CapiscIO",
  "url": "https://example.com/agent",
  "version": "1.0.0",
  "protocolVersion": "0.2.0",
  "provider": {
    "organization": "My Company",
    "url": "https://mycompany.com"
  },
  "capabilities": {
    "streaming": false,
    "pushNotifications": false,
    "stateTransitionHistory": false
  },
  "skills": [
    {
      "id": "greeting",
      "name": "Greeting",
      "description": "Returns a friendly greeting",
      "tags": ["communication", "social"],
      "examples": ["Say hello", "Greet the user"]
    }
  ],
  "authentication": {
    "schemes": ["none"]
  }
}

Re-run validation:

capiscio validate agent-card.json

✅ A2A AGENT VALIDATION PASSED

┌───────────────────────────────────────┐
│  Compliance Score:  95/100            │
│  Trust Score:       0/100 (unsigned)  │
│  Availability:      N/A               │
└───────────────────────────────────────┘

✓ All required fields present
✓ Protocol version valid
✓ Skills properly defined
✓ Provider information complete
✓ Authentication scheme specified

📊 Improvements:
  +20 points from adding provider info
  + 5 points from adding skill examples

Score improved from 75 to 95!

Trust Score¶

The Trust Score measures cryptographic verification of the agent's identity.

What's Checked¶

Without Signatures¶

Without a JWS signature, your trust score will be 0:

capiscio validate agent-card.json

⚠️  Trust Score: 0/100

TRUST WARNINGS:
  ⚠️  UNSIGNED_CARD: Agent card is not signed
  ⚠️  NO_KEY_ID: No key ID (kid) found

💡 To sign your agent card, see the Security Quickstart

With Signatures¶

To get a trust score, your agent card needs to be signed. We'll cover this in the Security Quickstart.

For now, if you're just validating (not deploying to production), you can skip signature verification:

capiscio validate agent-card.json --skip-signature

Availability Score¶

The Availability Score measures whether your agent's endpoint is reachable and responds correctly.

What's Checked¶

Testing Availability¶

Use --test-live to test a real endpoint:

capiscio validate https://real-agent.example.com/agent-card.json --test-live

✅ LIVE ENDPOINT TEST

┌───────────────────────────────────────┐
│  Availability Score:  100/100         │
└───────────────────────────────────────┘

✓ Endpoint reachable (245ms)
✓ Valid JSON response
✓ Correct content-type: application/json
✓ A2A protocol response verified

❌ LIVE ENDPOINT TEST FAILED

┌───────────────────────────────────────┐
│  Availability Score:  0/100           │
└───────────────────────────────────────┘

❌ Endpoint unreachable: connection timeout
💡 Verify the URL is correct and accessible

This sends an actual A2A message to the agent and verifies the response.

capiscio validate agent-card.json --schema-only

Production Readiness¶

CapiscIO determines if your agent is production ready based on these thresholds:

Check production readiness in JSON output:

capiscio validate agent-card.json --json | jq '.scoringResult.productionReady'

Or use strict mode (which enforces these thresholds):

capiscio validate agent-card.json --strict

✅ PRODUCTION READY

┌───────────────────────────────────────┐
│  Compliance:    98/100  ✓ (≥95)       │
│  Trust:         85/100  ✓ (≥60)       │
│  Availability:  95/100  ✓ (≥80)       │
└───────────────────────────────────────┘

🚀 Agent is production ready!

❌ NOT PRODUCTION READY

┌───────────────────────────────────────┐
│  Compliance:    75/100  ❌ (need ≥95) │
│  Trust:          0/100  ❌ (need ≥60) │
│  Availability:  N/A                   │
└───────────────────────────────────────┘

⚠️  Issues to fix:
  - Add missing optional fields for compliance
  - Sign agent card for trust verification

Exit code: 1

Issue Severity Levels¶

When issues are found, they have severity levels:

Common Issues and Fixes¶

Reading JSON Reports¶

For detailed analysis, use JSON output with jq:

capiscio validate agent-card.json --json | jq '.scoringResult'

{
  "complianceScore": 85,
  "trustScore": 0,
  "availabilityScore": null,
  "productionReady": false,
  "overallGrade": "B"
}

capiscio validate agent-card.json --json | jq '.issues[]'

{
  "code": "MISSING_OPTIONAL_FIELD",
  "severity": "warning",
  "message": "Missing 'provider.url' field",
  "path": "$.provider.url",
  "suggestion": "Add organization URL for better trust"
}

capiscio validate agent-card.json --json | jq '.scoringResult.productionReady'

false

What's Next?¶

You now understand:

• The three-dimensional scoring system
• What each score measures
• How to interpret issues
• How to improve your scores

Let's wrap up with next steps and resources!