Remote Inference & Tool Execution

Quick Access: Automation Sessions | Templates | Remote Servers

LangMart provides secure remote environments for executing LLM inference with MCP (Model Context Protocol) tool support. This enables AI models to interact with external tools like file systems, web browsers, and code execution in isolated, secure containers.

What You Get

Remote LLM Inference

Execute AI model requests in a secure, managed environment:

• Any Model: Use models from OpenAI, Anthropic, Google, Groq, and 20+ providers
• Streaming Support: Real-time response streaming
• Context Management: Maintain conversation state across requests
• Unified API: Same OpenAI-compatible API regardless of provider

MCP Tool Execution

AI models can invoke tools in a sandboxed environment:

Secure Execution Environment

Your sessions run in isolated Type 3 Gateways:

• Containerized: Each session runs in its own Docker container
• Isolated: No access to host system or other users' data
• Monitored: All tool executions are logged for transparency
• Temporary: Workspace is cleaned after session ends

How It Works

┌─────────────────────────────────────────────────────────────┐
│                    Your Application                          │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                   LangMart Platform                          │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐     │
│  │ Type 1/0    │───▶│ Type 3      │───▶│ AI Model    │     │
│  │ Gateway     │    │ Gateway     │    │ (Provider)  │     │
│  │ (API)       │    │ (Tools)     │    │             │     │
│  └─────────────┘    └─────────────┘    └─────────────┘     │
│                           │                                  │
│                           ▼                                  │
│                    ┌─────────────┐                          │
│                    │ Secure      │                          │
│                    │ Container   │                          │
│                    │ (Tools Run) │                          │
│                    └─────────────┘                          │
└─────────────────────────────────────────────────────────────┘

1. You send a request to the LangMart API
2. Request is routed to an available Type 3 Gateway
3. AI model processes your request and decides if tools are needed
4. Tools execute in a secure container if called
5. Results return through the same path

Getting Started

Using the Chat Interface

The easiest way to use remote inference with tools:

1. Go to Chat in the dashboard
2. Select Remote Chat mode (toggle in the top bar)
3. Choose a model with tool support
4. Start chatting - the AI can now use tools automatically

Using the API

Make requests with tool support enabled:

curl -X POST https://api.langmart.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4-20250514",
    "messages": [
      {"role": "user", "content": "Search for the latest news about AI"}
    ],
    "tools": "auto"
  }'

Session Management

Sessions maintain state across multiple requests:

# Create a session
POST /api/automation/sessions
{
  "name": "My Research Session",
  "model": "anthropic/claude-sonnet-4-20250514",
  "gateway_id": "optional-specific-gateway"
}

# Send messages to the session
POST /api/automation/sessions/{session_id}/messages
{
  "content": "Analyze the codebase in /workspace"
}

Available Tools

When using remote inference, models can access these MCP tools:

Registry Tools

• list_models - Browse available AI models
• list_providers - View supported providers
• get_model_info - Get detailed model information

Web Tools

• web_search - Search the internet
• fetch_page - Retrieve web page content
• extract_content - Parse and extract structured data

File Tools

• read_file - Read file contents
• write_file - Create or update files
• list_directory - Browse directories

Code Execution

• run_python - Execute Python code
• run_javascript - Execute JavaScript
• run_shell - Run shell commands

Use Cases

What's Next: Agent Workflows

This remote inference foundation enables advanced Agent Workflow Automation - where you can orchestrate complex multi-step tasks with:

• Chained tool executions
• Conditional logic and branching
• Scheduled and triggered workflows
• Multi-agent collaboration

Agent workflow documentation will be available in our Advanced Documentation section (coming soon).

Best Practices

Security

• Use sessions for sensitive work (isolated per session)
• Review tool outputs before using in production
• Set appropriate tool restrictions for your use case

Performance

• Reuse sessions for related tasks (maintains context)
• Choose models appropriate for tool use (Claude, GPT-4 recommended)
• Monitor session status for long-running tasks

Cost Management

• Tool executions may add to inference costs (additional tokens)
• Use streaming to monitor response progress
• Set up cost alerts for automation workloads

Quick Access Links

Related Topics

• Inference Sessions - Detailed session management
• Templates - Pre-configured session templates
• Chat Guide - Using tools in chat interface