Organization Setup for Inference Providers

This guide covers advanced organization configuration for Inference Providers, including settings optimized for serving members at scale.

Organization Types

When setting up your organization, consider your provider type:

Core Settings

General Configuration

Navigate to Organization > Settings to configure:

Member Settings

Control how members join and interact:

Domain-Based Auto-Approval

Automatically approve members from specific domains:

{
  "auto_approve_domains": [
    "acme.com",
    "acme-research.edu"
  ]
}

API Configuration

Rate Limits

Set organization-wide rate limits:

Via API

curl -X PUT https://api.langmart.ai/api/organizations/<org_id> \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "rate_limit_requests_per_minute": 200,
    "rate_limit_tokens_per_minute": 500000,
    "max_concurrent_requests": 50
  }'

Spending Controls

Organization-Level Limits

Set overall spending caps to prevent unexpected costs:

curl -X PUT https://api.langmart.ai/api/organizations/<org_id> \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "spending_limit_monthly": 5000.00,
    "spending_limit_daily": 500.00,
    "spending_alert_threshold": 0.8
  }'

Default Member Limits

New members automatically receive these limits:

curl -X PUT https://api.langmart.ai/api/organizations/<org_id> \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "default_member_monthly_limit": 100.00,
    "default_member_daily_limit": 10.00
  }'

Limit Recommendations by Provider Type

Security Settings

PII Detection

Enable automatic detection of sensitive data:

{
  "pii_detection_enabled": true,
  "pii_detection_mode": "warn",  // or "block"
  "pii_types": ["email", "phone", "ssn", "credit_card"]
}

KeyVault Settings

Configure automatic API key protection:

{
  "keyvault_enabled": true,
  "keyvault_auto_restore": true
}

Audit Logging

Enable comprehensive audit trails:

Access Control

Role Definitions

Define what each role can do:

Custom Roles (Enterprise)

Create custom roles for specific needs:

{
  "name": "Department Admin",
  "permissions": [
    "members:read",
    "members:invite",
    "usage:read",
    "connections:read"
  ]
}

Multi-Department Setup

For organizations with multiple departments or teams:

Department Tags

Tag members by department for reporting:

curl -X PUT https://api.langmart.ai/api/organizations/<org_id>/members/<user_id> \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": {
      "department": "Engineering",
      "cost_center": "ENG-001"
    }
  }'

Department-Specific Limits

Set different limits per department:

# Engineering: Higher limits
curl -X PUT https://api.langmart.ai/api/organizations/<org_id>/spending-limits \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "department": "Engineering",
    "monthly_limit": 500.00,
    "daily_limit": 50.00
  }'

Model Access Configuration

Model Categories

Organize available models by category:

Model Restrictions

Limit access to specific models:

{
  "allowed_models": [
    "openai/gpt-4o",
    "openai/gpt-3.5-turbo",
    "anthropic/claude-3.5-sonnet"
  ],
  "blocked_models": [
    "openai/o1-preview"  // Too expensive
  ]
}

Integration Settings

Webhook Configuration

Receive notifications for key events:

{
  "webhooks": [
    {
      "url": "https://your-server.com/webhook",
      "events": [
        "member.joined",
        "member.left",
        "spending_limit.reached",
        "connection.failed"
      ],
      "secret": "your-webhook-secret"
    }
  ]
}

SSO Integration (Enterprise)

Configure Single Sign-On:

Monitoring & Alerts

Alert Configuration

Set up alerts for key metrics:

Dashboard Customization

Configure what admins see on the dashboard:

• Real-time usage
• Cost by model
• Top users
• Error rates

Backup & Export

Data Export

Export organization data:

# Export usage data
curl -X GET "https://api.langmart.ai/api/organizations/<org_id>/export/usage?format=csv&start_date=2024-01-01" \
  -H "Authorization: Bearer <your-api-key>"

# Export member list
curl -X GET "https://api.langmart.ai/api/organizations/<org_id>/export/members?format=csv" \
  -H "Authorization: Bearer <your-api-key>"

Audit Log Export

Export audit logs for compliance:

curl -X GET "https://api.langmart.ai/api/admin/audit-logs/export?org_id=<org_id>&format=json" \
  -H "Authorization: Bearer <your-api-key>"

Best Practices

For Cloud Providers

1. Start with member-pays to manage risk
2. Set conservative initial limits
3. Monitor usage patterns before increasing
4. Use connection pools for reliability

For Enterprise IT

1. Map departments to cost centers
2. Enable PII detection
3. Set up SSO integration
4. Configure audit logging

For Research Institutions

1. Track usage by grant/project
2. Set per-project limits
3. Enable usage reports
4. Configure auto-approval for .edu domains

Troubleshooting

Members Not Seeing Models

1. Verify connection scope is "organization"
2. Check member is active (not pending)
3. Verify model is not in blocked list
4. Check member hasn't exceeded limits

High Costs

1. Review top users in usage dashboard
2. Check for runaway automation
3. Implement stricter daily limits
4. Consider member-pays for expensive models

Performance Issues

1. Check rate limits aren't too restrictive
2. Verify connection pool is configured
3. Monitor provider status pages
4. Review error logs for patterns

Next Steps

• Member Management - Managing users
• Billing Models - Billing configuration
• Connection Pools - High availability
• Usage Analytics - Monitoring