Usage Analytics for Inference Providers

Comprehensive analytics and reporting tools to monitor your inference service, track costs, and understand member usage patterns.

Analytics Dashboard

Overview Metrics

The dashboard provides real-time visibility into:

Time Ranges

View analytics for different periods:

• Today
• Last 7 days
• Last 30 days
• Custom range

Usage Breakdown

By Member

Track individual member consumption:

curl -X GET "https://api.langmart.ai/api/organizations/<org_id>/usage?group_by=user&start_date=2024-01-01&end_date=2024-01-31" \
  -H "Authorization: Bearer <your-api-key>"

Response:

{
  "success": true,
  "usage": {
    "by_user": [
      {
        "user_id": "uuid-1",
        "email": "[email protected]",
        "requests": 5000,
        "input_tokens": 2000000,
        "output_tokens": 500000,
        "cost_usd": 125.50,
        "avg_latency_ms": 450
      },
      {
        "user_id": "uuid-2",
        "email": "[email protected]",
        "requests": 2000,
        "input_tokens": 800000,
        "output_tokens": 200000,
        "cost_usd": 50.25,
        "avg_latency_ms": 380
      }
    ]
  }
}

By Model

Understand which models are most used:

curl -X GET "https://api.langmart.ai/api/organizations/<org_id>/usage?group_by=model" \
  -H "Authorization: Bearer <your-api-key>"

Response:

{
  "success": true,
  "usage": {
    "by_model": [
      {
        "model": "openai/gpt-4o",
        "requests": 8000,
        "input_tokens": 3000000,
        "output_tokens": 800000,
        "cost_usd": 200.00
      },
      {
        "model": "anthropic/claude-3.5-sonnet",
        "requests": 3000,
        "input_tokens": 1500000,
        "output_tokens": 400000,
        "cost_usd": 75.00
      }
    ]
  }
}

By Department

For organizations with department tagging:

curl -X GET "https://api.langmart.ai/api/organizations/<org_id>/usage?group_by=department" \
  -H "Authorization: Bearer <your-api-key>"

By Billing Mode

Compare org-pays vs member-pays usage:

curl -X GET "https://api.langmart.ai/api/organizations/<org_id>/usage?group_by=billing_mode" \
  -H "Authorization: Bearer <your-api-key>"

Cost Analysis

Cost Insights

Get detailed cost breakdown:

curl -X GET https://api.langmart.ai/api/account/cost-insights \
  -H "Authorization: Bearer <your-api-key>"

Response:

{
  "success": true,
  "insights": {
    "period": "last_30_days",
    "total_cost": 450.00,
    "cost_by_category": {
      "chat_completions": 380.00,
      "embeddings": 50.00,
      "image_generation": 20.00
    },
    "cost_trend": {
      "direction": "up",
      "percentage": 15,
      "previous_period": 391.30
    },
    "top_cost_drivers": [
      {"model": "openai/gpt-4o", "cost": 200.00},
      {"model": "anthropic/claude-3.5-sonnet", "cost": 100.00}
    ],
    "recommendations": [
      "Consider GPT-4o-mini for lower-complexity tasks",
      "Enable caching for repeated queries"
    ]
  }
}

Budget Tracking

Monitor spending against limits:

curl -X GET "https://api.langmart.ai/api/organizations/<org_id>/usage?include_budget=true" \
  -H "Authorization: Bearer <your-api-key>"

Response:

{
  "success": true,
  "budget": {
    "monthly_limit": 1000.00,
    "current_spend": 450.00,
    "percentage_used": 45,
    "projected_spend": 650.00,
    "days_remaining": 16
  }
}

Request Logs

Viewing Logs

Access detailed request history:

curl -X GET "https://api.langmart.ai/api/account/request-logs?limit=50" \
  -H "Authorization: Bearer <your-api-key>"

Response:

{
  "success": true,
  "logs": [
    {
      "id": "log-123",
      "timestamp": "2024-01-15T10:30:00Z",
      "user_id": "uuid-1",
      "model": "openai/gpt-4o",
      "status": "success",
      "input_tokens": 500,
      "output_tokens": 150,
      "cost_usd": 0.02,
      "latency_ms": 450,
      "connection_id": "conn-456"
    }
  ]
}

Log Filtering

Filter logs by various criteria:

curl -X GET "https://api.langmart.ai/api/account/request-logs?user_id=uuid-1&model=openai/gpt-4o&status=error" \
  -H "Authorization: Bearer <your-api-key>"

Enhanced Log Details

Get full request/response data:

curl -X GET "https://api.langmart.ai/api/account/request-logs/<log_id>/details" \
  -H "Authorization: Bearer <your-api-key>"

Error Tracking

Error Analytics

Monitor error patterns:

curl -X GET https://api.langmart.ai/api/account/errors \
  -H "Authorization: Bearer <your-api-key>"

Response:

{
  "success": true,
  "errors": {
    "total_errors": 150,
    "error_rate": 0.02,
    "by_type": [
      {"type": "rate_limit", "count": 80},
      {"type": "invalid_request", "count": 50},
      {"type": "timeout", "count": 20}
    ],
    "by_model": [
      {"model": "openai/gpt-4o", "errors": 100},
      {"model": "groq/llama-3.3-70b", "errors": 50}
    ]
  }
}

Error Types

Reports & Exports

Usage Report

Generate comprehensive usage reports:

curl -X GET "https://api.langmart.ai/api/organizations/<org_id>/export/usage?format=csv&start_date=2024-01-01&end_date=2024-01-31" \
  -H "Authorization: Bearer <your-api-key>"

Report Formats

Scheduled Reports

Set up automated report delivery:

curl -X POST https://api.langmart.ai/api/organizations/<org_id>/reports/schedule \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Weekly Usage Report",
    "frequency": "weekly",
    "day": "monday",
    "format": "pdf",
    "recipients": ["[email protected]"],
    "include": ["usage_summary", "top_users", "cost_breakdown"]
  }'

Invoice Generation

Generate invoices for chargeback:

curl -X POST https://api.langmart.ai/api/billing/invoices/generate \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "period_start": "2024-01-01",
    "period_end": "2024-01-31",
    "group_by": "department"
  }'

Alerts & Notifications

Setting Up Alerts

Create alerts for key metrics:

curl -X POST https://api.langmart.ai/api/account/alerts \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "High Usage Alert",
    "type": "spending",
    "condition": "daily_spend > 100",
    "action": "email",
    "recipients": ["[email protected]"]
  }'

Alert Types

Alert Conditions

Real-Time Monitoring

WebSocket Feed

Subscribe to real-time usage events:

const ws = new WebSocket('wss://api.langmart.ai/ws/analytics');
ws.send(JSON.stringify({
  type: 'subscribe',
  org_id: '<org_id>',
  events: ['request', 'error', 'cost_update']
}));

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log('Analytics event:', data);
};

Event Types

API Reference

List Available Metrics

curl -X GET https://api.langmart.ai/api/analytics/metrics \
  -H "Authorization: Bearer <your-api-key>"

Custom Queries

For advanced analysis:

curl -X POST https://api.langmart.ai/api/analytics/query \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "metrics": ["requests", "tokens", "cost"],
    "dimensions": ["user", "model", "day"],
    "filters": {
      "start_date": "2024-01-01",
      "end_date": "2024-01-31",
      "billing_mode": "org_pays"
    },
    "sort": "cost",
    "limit": 100
  }'

Best Practices

Monitoring Strategy

1. Daily: Check error rates and unusual spikes
2. Weekly: Review top users and model usage
3. Monthly: Generate reports, assess costs
4. Quarterly: Trend analysis, budget planning

Alert Strategy

Optimization Tips

1. Identify expensive models: Switch to alternatives where possible
2. Find power users: Work with them on efficiency
3. Spot patterns: Time-of-day, day-of-week trends
4. Reduce errors: Fix common issues to avoid retries

Troubleshooting

Missing Data

Export Failures

Next Steps

• Billing Models - Cost management
• Member Management - User analytics
• Connection Pools - Performance monitoring