Use Case 036: Cost Insights Dashboard & Billing Report

Overview

Implementation status (agent-identity release, June 2026). Implemented: the pre-aggregated SpendRollup (daily grain) fed by an hourly Hangfire job (spend-rollup), the /insights dashboard (gated by cost.insights.view.tenant / .agency), and CSV/JSON billing-report export. OIDC budget-viewer access is wired end-to-end with a development-only mock IdP; see docs/oidc-azure-budget-viewer.md. API at /api/insights.

Description

This use case describes the read-only insights dashboard (/insights/*) and the billing report export introduced in the agent-identity release. The dashboard is a finance-and-leadership surface (the CTO / budget-hawk persona) that answers business questions about LLM spend — by provider, tenant, application, template, owner, and subscription — and is deliberately distinct from the AM admin UI. It is backed entirely by the pre-aggregated SpendRollup table (never the raw LlmCallLedger), so heavy dashboard traffic never touches the hot enforcement path. The companion billing report (POST /api/insights/billing-report) emits a configurable-period CSV/JSON extract that accounting can consume; it is explicitly not an invoice-reconciliation tool.

Authentication for /insights/* reuses the existing OIDC integration against two new roles — TenantBudgetViewer (scoped to one tenant) and AgencyBudgetViewer (cross-tenant) — neither of which requires an AM admin grant. The /insights/* routes register with a distinct authentication scheme from the admin UI cookie auth (they issue the OIDC challenge directly). The SDK master design is riptide-sdk/docs/plans/AGENT-IDENTITY-ARCHITECTURE.md; the Application Manager plan is docs/internal/agent-identity-plan.md (Section 4 for the dashboard, Section 7 for the billing report, §3.8 for the Coverage panel / scope honesty). These are planned features and not yet built.

Actors

Preconditions

1. The agent-identity / cost-governance plane is deployed; SpendRollup, LlmCallLedger, and BudgetCap tables exist (EF migration applied).
2. At least one OIDC identity provider is enabled and configured to emit a role claim mapping to TenantBudgetViewer and/or AgencyBudgetViewer.
3. The new capabilities (cost.insights.view.tenant, cost.insights.view.agency, cost.report.generate) are registered and mapped to roles (see UC-038).
4. The SpendRollupJob has run at least once (hourly Hangfire job) so SpendRollup contains rows; an empty rollup yields a cold-start empty-state dashboard.
5. (For billing report) The caller holds cost.report.generate.

Postconditions

Success Postconditions

1. The viewer sees an OIDC-authenticated dashboard scoped to their role (single tenant or cross-tenant), backed by SpendRollup.
2. Drill-down views resolve to the (tenant, provider, modelDeployment, application, template, owner, day) grain — never individual agent ids.
3. The Coverage panel renders AM-governed spend, the provider-invoice total (if entered), the computed gap, and the explicit not-covered list (§3.8).
4. A billing report is produced for the requested period and scope as CSV or JSON with the fixed column set.
5. Dashboard access and billing-report generation are recorded in the audit trail (AuditLog).

Failure Postconditions

1. A viewer without a budget-viewer role is denied (403) and redirected to the OIDC challenge; no spend data is exposed.
2. A TenantBudgetViewer requesting another tenant's scope receives an empty/filtered result, never cross-tenant data.
3. A billing-report request without cost.report.generate is rejected (403); no file is produced.
4. If SpendRollup is empty, the dashboard shows an empty state rather than erroring.

Primary Flow Sequence

sequenceDiagram
    actor Viewer as Budget Viewer
    participant Web as Web /insights/*
    participant OIDC as OIDC Provider
    participant API as InsightsController (API)
    participant MR as MediatR
    participant Roll as SpendRollup (read)

    Viewer->>Web: GET /insights/overview
    Web->>Web: Check insights auth scheme (cookie absent)
    Web->>OIDC: Challenge (redirect)
    OIDC-->>Web: Auth code -> role claims
    Web->>Web: Resolve TenantBudgetViewer / AgencyBudgetViewer + scope
    Web->>API: GET /api/insights/overview (scope)
    API->>API: Enforce cost.insights.view.{tenant|agency}
    API->>MR: GetInsightsOverview(scope)
    MR->>Roll: Aggregate SpendRollup rows (no ledger touch)
    Roll-->>MR: Totals, top providers, burn rate
    MR-->>API: Overview DTO
    API-->>Web: 200 overview JSON
    Web-->>Viewer: Render Overview (Bootstrap + Tabler)

UC-036a: View Insights Overview (OIDC Viewer Role)

Triggers

• A budget viewer navigates to /insights or /insights/overview.

Basic Flow

1. The viewer requests /insights/overview.
2. The /insights/* auth scheme finds no valid session and issues an OIDC challenge (distinct from the admin cookie scheme).
3. The OIDC provider authenticates the viewer; AM maps the role claim to TenantBudgetViewer (single tenant) or AgencyBudgetViewer (cross-tenant) and records the scope.
4. The Web tier calls GET /api/insights/overview, passing the resolved scope.
5. InsightsController enforces cost.insights.view.tenant or cost.insights.view.agency.
6. GetInsightsOverview aggregates SpendRollup rows (totals, month-to-date spend, top providers/applications, current burn rate) — never the raw ledger.
7. The dashboard renders the Overview with cards and charts and a left-nav list of the business-question views.

Alternative Flows

• A1: No budget-viewer role — OIDC succeeds but the role claim maps to neither viewer role; access denied (403); the viewer cannot enter the dashboard.
• A2: Cold start (empty rollup) — SpendRollup has no rows for the scope; an empty state is rendered ("No governed spend recorded yet") instead of an error.
• A3: Tenant viewer, agency data requested — a TenantBudgetViewer cannot widen scope; the cross-tenant nav items (e.g., By Tenant) are hidden and any direct request is filtered to their tenant.

UC-036b: Drill-Down Views

Triggers

• The viewer selects a business-question view: By Provider, By Tenant (agency-level), By Application, By Template, By Owner, By Subscription, Period-over-period.

Basic Flow

1. The viewer selects a drill-down (e.g., By Provider).
2. The Web tier calls the matching endpoint (GET /api/insights/by-provider, etc.) with the viewer's scope and optional period filter.
3. The handler aggregates SpendRollup at the requested dimension, stopping at the (tenant, provider, modelDeployment, application, template, owner, day) grain.
4. The view renders a sortable table plus a trend chart; By Provider also shows inline subscription context.
5. The viewer may pivot to another dimension or change the period; each pivot re-queries the rollup.

Alternative Flows

• A1: Drill past the supported grain — there is no path to individual agent ids; that grain is intentionally unavailable (agents are too ephemeral; forensic agent-level detail lives in the audit log).
• A2: By Tenant for a tenant viewer — hidden/disabled; only AgencyBudgetViewer sees cross-tenant rollups.
• A3: Period-over-period with insufficient history — prior-period columns render as "n/a" rather than zero.

UC-036c: Caps & Burn-Rate Projections

Triggers

• The viewer opens the Caps & burn rates view.

Basic Flow

1. The viewer opens Caps & burn rates.
2. The Web tier calls GET /api/insights/caps-status.
3. GetCapsStatus returns every applicable BudgetCap in scope with current-period consumption, limit, percent consumed, and a projected end-of-period spend derived from the current burn rate.
4. Caps near or over threshold are visually flagged (warning/danger badges).
5. The viewer reviews which caps are trending toward exhaustion before period end.

Alternative Flows

• A1: No caps configured in scope — the view shows "No caps configured" (note: a Top-tier cap is required at tenant creation, so this is rare).
• A2: Projection unstable (sparse data) — projection column shows "insufficient data" instead of an extrapolated figure.

UC-036d: Coverage Panel / Scope Honesty

Triggers

• The viewer opens the Coverage panel (a tab on the Overview).

Basic Flow

1. The viewer opens the Coverage panel.
2. AM computes AM-governed spend this month: $X from SpendRollup.
3. If the viewer (or an admin) has entered the provider invoice total: $Y for the period, AM shows Gap: $Z = $Y − $X.
4. The panel renders the explicit list of what AM caps do not cover:
   • Direct provider API calls made with raw, non-AM-issued keys.
   • Pre-existing provider spend from before AM was deployed.
   • MCP servers that make their own LLM calls without going through the passthrough endpoints or LlmClient.
   • Provider-side internal line items: fine-tuning, custom-model hosting, evaluations, support costs.
5. The viewer interprets the gap as the un-migrated / un-governed surface and watches it close over time.

Alternative Flows

• A1: No invoice total entered — only $X is shown; the gap reads "invoice total not provided"; the not-covered list still renders.
• A2: Gap negative — if $X exceeds the entered $Y (e.g., stale invoice), the panel flags a data-entry warning rather than a negative gap.

sequenceDiagram
    actor Viewer as Budget Viewer
    participant Web as Web /insights/coverage
    participant API as InsightsController
    participant MR as MediatR
    participant Roll as SpendRollup

    Viewer->>Web: Open Coverage panel
    Web->>API: GET /api/insights/overview?include=coverage
    API->>MR: GetInsightsOverview(scope, coverage=true)
    MR->>Roll: Sum governed spend this month ($X)
    Roll-->>MR: $X
    MR-->>API: $X + entered invoice $Y + not-covered list
    API-->>Web: Coverage payload (X, Y, Z, exclusions)
    Web-->>Viewer: Render $X / $Y / Gap $Z + exclusions

UC-036e: Generate Billing Report (CSV/JSON)

Triggers

• An accounting user (or automation) holding cost.report.generate requests a billing report for a period and scope.

Basic Flow

1. The caller submits POST /api/insights/billing-report with a period (start/end), output format (csv or json), and a scope filter (any of tenant / provider / providerAccount / modelDeployment).
2. The API enforces cost.report.generate.
3. GenerateBillingReport aggregates SpendRollup rows (rolling up to the requested scope) for the period.
4. AM produces rows with the fixed column set: tenant, provider, providerAccount, modelDeployment, model, application, template, owner, period, callCount, inputTokens, outputTokens, cachedTokens, computedCost, pricingVersion.
5. The report is returned in the requested format; the report shape is identical regardless of provider (cloud, local, or any future enforcement mechanism).
6. The generation event is recorded in AuditLog.

Not reconciliation. The billing report is an accounting-input extract. AM does not ingest provider invoices, does not perform three-way reconciliation, and does not arbitrate bill-of-materials disputes. Those are downstream of this report.

Alternative Flows

• A1: Missing cost.report.generate — request rejected (403); no file produced.
• A2: Empty period — a valid report with zero data rows (header only) is returned, not an error.
• A3: Scope a tenant viewer cannot see — filtered to the caller's permitted scope; cross-tenant rows are excluded.
• A4: Local/self-hosted deployment rows — produced with PricingAuthority = Internal pricing; same row shape as cloud rows.

sequenceDiagram
    actor Acct as Accounting
    participant API as InsightsController
    participant MR as MediatR
    participant Roll as SpendRollup
    participant Audit as AuditLog

    Acct->>API: POST /api/insights/billing-report (period, scope, format)
    API->>API: Enforce cost.report.generate
    API->>MR: GenerateBillingReport(period, scope, format)
    MR->>Roll: Aggregate rows for period + scope
    Roll-->>MR: Rolled-up rows
    MR->>MR: Project fixed column set
    MR-->>API: CSV / JSON payload
    API->>Audit: Record report generation (actor, period, scope)
    API-->>Acct: 200 report (text/csv or application/json)

API Endpoints

// POST /api/insights/billing-report
// Request
{
  "period": { "start": "2026-05-01", "end": "2026-05-31" },
  "format": "json",
  "scope": {
    "tenantId": "5f7a1c2e-0b3d-4e6f-9a01-2c3d4e5f6a7b",
    "providerId": "Anthropic",
    "providerAccountId": null,
    "modelDeploymentId": null
  }
}

// Response 200
{
  "period": { "start": "2026-05-01", "end": "2026-05-31" },
  "generatedAt": "2026-06-09T14:05:00Z",
  "rows": [
    {
      "tenant": "Acme Agency",
      "provider": "Anthropic",
      "providerAccount": "acme-anthropic-prod",
      "modelDeployment": "claude-sonnet-cloud",
      "model": "claude-sonnet-4",
      "application": "support-bot",
      "template": "chat.customer-support",
      "owner": "jdray@acme.gov",
      "period": "2026-05",
      "callCount": 18422,
      "inputTokens": 14233012,
      "outputTokens": 3120044,
      "cachedTokens": 982110,
      "computedCost": 412.77,
      "pricingVersion": "anthropic-2026-04"
    }
  ]
}

Business Rules

Data Requirements

SpendRollup (read source; owned by UC-034) — fields consumed by this use case:

Billing report row (projection, not a stored entity):

Security Considerations

• Authentication: /insights/* authenticates via the existing OIDC integration under a dedicated scheme, separate from admin cookie auth; no anonymous access.
• Authorization / capabilities: view endpoints gate on cost.insights.view.tenant or cost.insights.view.agency; the billing report gates on cost.report.generate. A TenantBudgetViewer is hard-scoped to one tenant; cross-tenant nav and data are unavailable to it.
• Data protection: the dashboard never exposes agent-level detail or raw ledger rows; only the pre-aggregated SpendRollup grain is reachable. The not-covered list keeps the trust story honest about un-governed spend.
• Audit: dashboard access and every billing-report generation are recorded in AuditLog (actor, scope, period). No secrets or provider credentials are ever surfaced in any view.

Testing Scenarios

Related Use Cases

• UC-034 — SpendRollup Aggregation (data source for all dashboard views and the billing report)
• UC-037 — Spend Anomaly Detection (feeds the Anomalies panel and GET /api/insights/anomalies)
• UC-038 — Agent Cost Governance RBAC (defines cost.insights.view.tenant/.agency and cost.report.generate)
• UC-032 — Budget Cap Lattice Enforcement (caps surfaced in the Caps & burn rates view)
• UC-018 — External Identity Provider Integration (OIDC integration reused for viewer roles)

Revision History