Question: GPU/Docker Execution and Tool/MCP Integration Support

[passing2961]

Hi,

I’m wondering whether OpenEvolve supports LLM interaction with an external execution environment such as a GPU cluster or a Docker-based server. For example, in a task like “discovering an optimized mixture-of-experts architecture for mathematical reasoning,” the system would need to fine-tune or evaluate candidate model architectures on a GPU environment. Does OpenEvolve support such GPU-backed execution, and is it also possible to call external tools or MCP servers during the evolutionary process?

Thanks!

[codelion]

GPU/Docker Execution

LLM Operations

OpenEvolve supports any OpenAI-compatible API endpoint via the api_base configuration. You can point to:

• Self-hosted GPU servers running vLLM, TGI, Ollama, or similar
• Docker containers with OpenAI-compatible APIs
• Cloud GPU endpoints (Together AI, Fireworks, etc.)

Example config:

llm:
  api_base: "http://your-gpu-server:8000/v1"
  api_key: "your-key"
  models:
    - name: "your-model"
      weight: 1.0

Evaluation on GPU/Docker

The evaluator is fully customizable. Your evaluator.py can call remote GPU resources. For your MoE architecture discovery use case:

def evaluate(program_path):
    # Read the candidate architecture
    with open(program_path) as f:
        architecture_code = f.read()

    # Submit to your GPU cluster/Docker for training
    job_id = submit_training_job(architecture_code, gpu_cluster_url)
    results = wait_for_job(job_id)

    return {
        "accuracy": results["accuracy"],
        "efficiency": results["params"] / results["flops"],
        "combined_score": compute_final_score(results)
    }

Configure longer timeouts for training jobs:

evaluator:
  timeout: 3600  # 1 hour for training jobs

Tool/MCP Integration

You can use https://github.com/algorithmicsuperintelligence/optillm as a proxy to enable tool calling and MCP server integration with any model. OptiLLM sits between OpenEvolve and your LLM, adding capabilities like:

• Tool/function calling support
• MCP server connections
• Various inference optimization techniques

Setup:

1. Configure MCP servers in ~/.optillm/mcp_config.json:

{
  "servers": {
    "my-gpu-tools": {
      "transport": "sse",
      "url": "http://your-mcp-server:8080/sse",
      "headers": {
        "Authorization": "Bearer ${API_TOKEN}"
      }
    }
  }
}

2. Point OpenEvolve to OptiLLM and use the mcp- prefix:

llm:
  api_base: "http://localhost:8000/v1"  # OptiLLM proxy
  models:
    - name: "mcp-gpt-4o"  # mcp prefix enables MCP plugin
      weight: 1.0

This allows the LLM to call external tools and MCP servers during code generation.