Surf vs MCP vs Web Scraping

Overview

When you want an AI agent to interact with a website — read products, submit forms, fetch data — you broadly have three options today:

Feature Comparison

A direct feature-by-feature breakdown. Numbers are medians measured across real agent runs where noted.

Surf vs MCP

MCP is excellent — and it's not a competitor. Here's how they differ:

import Anthropic from '@anthropic-ai/sdk'
import { createSurfClient } from '@surfjs/client'
 
// MCP for local tools (filesystem, shell, etc.)
const mcp = new Anthropic.McpClient({ server: 'stdio://./my-mcp-server' })
 
// Surf for your web API
const surf = await createSurfClient('https://api.myapp.com')
const commands = await surf.discover()
 
// Both available in the same LLM call
const result = await anthropic.messages.create({
  tools: [...mcp.tools(), ...commands.asTools()],
  // ...
})

Surf vs Web Scraping

Scraping is the fallback when no structured API exists. If you control the server, Surf is strictly better on every metric that matters for agents.

The catch: scraping works on any site. Surf requires the site owner to add it. If you're building a general-purpose agent crawler, you'll need scraping as a fallback for sites that haven't adopted Surf yet.

When to Use What

Migration Path

Already using web scraping or MCP? You don't need to rip anything out. Surf can coexist as an additive layer.

From scraping → Surf

Add Surf to your most-scraped routes first. The agent will prefer the Surf endpoint (it's faster and more reliable) and fall back to scraping for everything else.

import express from 'express'
import { createSurf } from '@surfjs/core'
import { expressAdapter } from '@surfjs/express'
 
const app = express()
const surf = await createSurf()
 
// Start by surfing just your most-scraped routes
surf.command('search', async ({ q }) => {
  const results = await db.products.search(q)
  return results.map(p => ({ id: p.id, name: p.name, price: p.price }))
})
 
surf.command('getProduct', async ({ id }) => {
  return db.products.findById(id)
})
 
// Everything else still works as before
app.get('/products', legacyHandler)
app.use(expressAdapter(surf))
app.listen(3000)

From MCP → Surf (for web routes)

If you have an MCP server that wraps your web API with fetch() calls, consider moving those tools to Surf. You'll get auto-discovery, real-time events, and built-in auth — and your MCP server can stay for local tools.