cosmic-mcp

All servers

evalsmith-mcp

io.github.diegoperezg7/evalsmith-mcp ยท version 0.1.0

Community listing. Not installed or verified by cosmic-mcp. Read the source and its permissions before you run it.

Evaluate RAG retrieval, agent tool calls and prompt regressions with deterministic metrics.

https://github.com/diegoperezg7/evalsmith-mcp

Quick facts

  • Listing: Community
  • Published Oct 28, 10:31 UTC
  • Transport: stdio
  • Package: npm evalsmith-mcp 0.1.0
  • Listed by cosmic-mcp: Nov 9, 04:25 UTC

Vector search and RAG AI and LLMs Data and analytics

Setup snippets

Generated from the registry entry. cosmic-mcp has not run or reviewed this server; values in angle brackets are yours to fill in.

Run it locally (npm)

npx -y '[email protected]'

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "evalsmith-mcp": {
      "args": [
        "-y",
        "[email protected]"
      ],
      "command": "npx",
      "env": {
        "EVALSMITH_JUDGE_API_KEY": "<your evalsmith judge api key>",
        "EVALSMITH_JUDGE_BASE_URL": "<your evalsmith judge base url>",
        "EVALSMITH_JUDGE_MODEL": "<your evalsmith judge model>",
        "LOG_LEVEL": "<your log level>"
      }
    }
  }
}

Claude Code

claude mcp add 'evalsmith-mcp' -- npx -y '[email protected]'

Cursor (.cursor/mcp.json)

{
  "mcpServers": {
    "evalsmith-mcp": {
      "args": [
        "-y",
        "[email protected]"
      ],
      "command": "npx",
      "env": {
        "EVALSMITH_JUDGE_API_KEY": "<your evalsmith judge api key>",
        "EVALSMITH_JUDGE_BASE_URL": "<your evalsmith judge base url>",
        "EVALSMITH_JUDGE_MODEL": "<your evalsmith judge model>",
        "LOG_LEVEL": "<your log level>"
      }
    }
  }
}

VS Code (.vscode/mcp.json)

{
  "servers": {
    "evalsmith-mcp": {
      "args": [
        "-y",
        "[email protected]"
      ],
      "command": "npx",
      "env": {
        "EVALSMITH_JUDGE_API_KEY": "<your evalsmith judge api key>",
        "EVALSMITH_JUDGE_BASE_URL": "<your evalsmith judge base url>",
        "EVALSMITH_JUDGE_MODEL": "<your evalsmith judge model>",
        "LOG_LEVEL": "<your log level>"
      },
      "type": "stdio"
    }
  }
}

Related servers

  • Harness Verified domain
    Start Claude Code and Codex agents the person can watch, wait for them and read their answers.
  • docs-mcp Verified domain
    Remote MCP server for Tandem docs, install guides, SDKs, workflows, and agent setup help.
  • spotdb Verified domain
    Ephemeral data sandbox for AI workflows with guardrails and security
  • Adako: ad ops for AI assistants Verified domain
    Google, Meta, LinkedIn and ChatGPT Ads for AI assistants. Every change needs your approval.
  • Aidoo for Odoo Verified domain
    Query and update your Odoo ERP in plain language, under your own Odoo access rules.
  • EXTREMELY agency Verified domain
    Your agency of AI agents in your AI app: what agents did, approvals, requests, jobs and reports.