Zum Inhalt

Referenz des MCP-Servers

Der MCP-Server von Scrapling bietet siebzehn Werkzeuge zum Extrahieren von Daten und zur Bedienung des Browsers über das Model Context Protocol (MCP). Binden Sie Scrapling in Assistenten und KI-Agenten ein, um Daten per Gespräch zu extrahieren, mit Funktionen gegen Bot-Erkennung.

Starten Sie den MCP-Server mit diesem Befehl:

scrapling mcp

Oder importieren Sie die Serverklasse direkt:

from scrapling.core.ai import ScraplingMCPServer

server = ScraplingMCPServer()
server.serve(http=False, host="127.0.0.1", port=8000)

Um Ihre eigene ausführbare Datei eines Chromium-kompatiblen Browsers zu verwenden, übergeben Sie executable_path:

server = ScraplingMCPServer(executable_path="/path/to/chromium")

Antwortmodell

Die Werkzeuge zum Abrufen von Seiten und für HTTP-Anfragen geben diese Struktur zurück, mit einer Antwort pro Aufruf. Die Werkzeuge zur Sitzungsverwaltung verwenden die folgenden Modelle. browser_screenshot gibt ein Bild der Seite oder eines Elements zusammen mit der aktuellen URL als Bild- und Textblöcke zurück. browser_extract und browser_actions geben einfachen Text zurück. Die beiden Netzwerkwerkzeuge geben strukturiertes JSON über die folgenden Modelle zurück. browser_evaluate gibt das Ergebnis als JSON in einem Textblock zurück.

scrapling.core.ai.server.ResponseModel

Basisklassen: BaseModel


              flowchart TD
              scrapling.core.ai.server.ResponseModel[ResponseModel]

              

              click scrapling.core.ai.server.ResponseModel href "" "scrapling.core.ai.server.ResponseModel"
            

Datenstruktur mit den Antwortinformationen einer Anfrage.

status class-attribute instance-attribute

status = Field(
    description="The status code returned by the website."
)

content class-attribute instance-attribute

content = Field(
    description="The page content as Markdown, HTML, text, or an AI ARIA snapshot."
)

url class-attribute instance-attribute

url = Field(
    description="The URL given by the user that resulted in this response."
)

Modelle für Netzwerkanfragen

browser_network_requests gibt NetworkRequestsModel mit requests, next_cursor, has_more und dropped_count zurück. Jedes NetworkRequestInfo enthält id, url, method, resource_type und status. Zum Fortsetzen übergeben Sie next_cursor als after_id. has_more zeigt an, ob der aktuelle Verlauf weitere Treffer enthält.

browser_network_request gibt NetworkRequestModel mit request_id, part, data und note zurück. Zusammenfassungen und Header sind Objekte. Der vollständig gespeicherte Inhalt wird als Zeichenkette zurückgegeben, auch wenn die Antwort JSON enthält. note erklärt, ob Inhaltsdaten fehlen oder Header unvollständig sind.

scrapling.core.ai._network_formatting.NetworkRequestInfo

Basisklassen: BaseModel


              flowchart TD
              scrapling.core.ai._network_formatting.NetworkRequestInfo[NetworkRequestInfo]

              

              click scrapling.core.ai._network_formatting.NetworkRequestInfo href "" "scrapling.core.ai._network_formatting.NetworkRequestInfo"
            

Zusammenfassung einer aufgezeichneten Browser-Anfrage.

model_config class-attribute instance-attribute

model_config = {'populate_by_name': True}

id class-attribute instance-attribute

id = Field(validation_alias=AliasPath("meta", "network_id"))

url instance-attribute

url

method instance-attribute

method

resource_type class-attribute instance-attribute

resource_type = Field(
    validation_alias=AliasPath("meta", "resource_type")
)

status instance-attribute

status

scrapling.core.ai.server.NetworkRequestsModel

Basisklassen: BaseModel


              flowchart TD
              scrapling.core.ai.server.NetworkRequestsModel[NetworkRequestsModel]

              

              click scrapling.core.ai.server.NetworkRequestsModel href "" "scrapling.core.ai.server.NetworkRequestsModel"
            

Zusammenfassungen aufgezeichneter Anfragen mit Angaben zum seitenweisen Abruf.

requests instance-attribute

requests

next_cursor instance-attribute

next_cursor

has_more instance-attribute

has_more

dropped_count instance-attribute

dropped_count

scrapling.core.ai._network_formatting.NetworkRequestModel

Basisklassen: BaseModel


              flowchart TD
              scrapling.core.ai._network_formatting.NetworkRequestModel[NetworkRequestModel]

              

              click scrapling.core.ai._network_formatting.NetworkRequestModel href "" "scrapling.core.ai._network_formatting.NetworkRequestModel"
            

Ein vollständiger gespeicherter Teil einer Anfrage mit einem optionalen Hinweis.

request_id instance-attribute

request_id

part instance-attribute

part

data class-attribute instance-attribute

data = None

note class-attribute instance-attribute

note = None

Sitzungsmodelle

Modellklassen zur Verwaltung von Sitzungen:

scrapling.core.ai.server.SessionInfo

Basisklassen: SessionCreatedModel


              flowchart TD
              scrapling.core.ai.server.SessionInfo[SessionInfo]
              scrapling.core.ai.server.SessionCreatedModel[SessionCreatedModel]

                              scrapling.core.ai.server.SessionCreatedModel --> scrapling.core.ai.server.SessionInfo
                


              click scrapling.core.ai.server.SessionInfo href "" "scrapling.core.ai.server.SessionInfo"
              click scrapling.core.ai.server.SessionCreatedModel href "" "scrapling.core.ai.server.SessionCreatedModel"
            

Informationen über eine offene Browser-Sitzung.

created_at class-attribute instance-attribute

created_at = Field(
    description="ISO timestamp of when the session was created."
)

is_alive class-attribute instance-attribute

is_alive = Field(
    description="Whether the session is still alive and usable."
)

settings class-attribute instance-attribute

settings = Field(
    default_factory=dict,
    description="The effective settings this session was created with.",
)

session_id class-attribute instance-attribute

session_id = Field(
    description="The unique identifier of the session."
)

session_type class-attribute instance-attribute

session_type = Field(
    description="The type of the session: 'stealthy' or 'static'."
)

scrapling.core.ai.server.SessionCreatedModel

Basisklassen: BaseModel


              flowchart TD
              scrapling.core.ai.server.SessionCreatedModel[SessionCreatedModel]

              

              click scrapling.core.ai.server.SessionCreatedModel href "" "scrapling.core.ai.server.SessionCreatedModel"
            

Antwort beim Erstellen einer neuen Sitzung.

session_id class-attribute instance-attribute

session_id = Field(
    description="The unique identifier of the session."
)

session_type class-attribute instance-attribute

session_type = Field(
    description="The type of the session: 'stealthy' or 'static'."
)

scrapling.core.ai.server.SessionClosedModel

Basisklassen: BaseModel


              flowchart TD
              scrapling.core.ai.server.SessionClosedModel[SessionClosedModel]

              

              click scrapling.core.ai.server.SessionClosedModel href "" "scrapling.core.ai.server.SessionClosedModel"
            

Antwort beim Schließen einer Sitzung.

session_id class-attribute instance-attribute

session_id = Field(
    description="The unique identifier of the closed session."
)

message class-attribute instance-attribute

message = Field(description='A confirmation message.')

Klasse des MCP-Servers

Die Hauptklasse des MCP-Servers bietet diese Werkzeuge:

  • make_request: Ruft eine URL über HTTP ab, bildet einen Browser nach und schließt den Client automatisch. Unterstützt GET, POST, PUT und DELETE.
  • browser_fetch_once: Ruft eine URL mit einem Stealth-Browser ab, führt JavaScript aus, behandelt Cloudflare und schließt den Browser. Nutzen Sie eine dauerhafte Sitzung, wenn Sie danach Aktionen ausführen oder den Netzwerkverlauf lesen möchten.

  • browser_extract: Liest die aktuelle Seite oder ein Element als strukturierte Ansicht für KI, HTML, Markdown oder Text, ohne neu zu laden. Ergebnisse von Formularen und Aktionen bleiben erhalten. Die strukturierten Ansichten enthalten Rollen, Namen, Referenzen, Positionen und Größen der Elemente. Die Suche mit regulären Ausdrücken liefert passende Zeilen, den nahen Kontext und Elternknoten.

  • browser_actions: Verknüpft Mausbewegungen, Bewegen des Zeigers über Elemente, Klicks, Scrollen, Feldeingaben, Tastenkürzel, Antworten auf Dialoge und Wartezeiten in einem Aufruf. Akzeptieren oder schließen Sie Meldungen und Bestätigungen, oder geben Sie Text in Eingabedialoge ein. Sie können ein Suchfeld ausfüllen, die Suche absenden, auf Ergebnisse warten und eines öffnen, ohne einen Aufruf pro Schritt zu machen.

    Bewegungen und Klicks akzeptieren Selektoren, Referenzen aus strukturierten Ansichten oder Koordinaten. Das Werkzeug wartet auf Elemente und scrollt sie in den sichtbaren Bereich. Es unterstützt Bewegungen in Schritten, linke, rechte und mittlere Mausklicks, Doppelklicks und Scrollen in verschachtelten Bereichen.

    Halten Sie eine Maustaste gedrückt, um Elemente zu ziehen oder Schieberegler zu bewegen. Die KI kann die Seite bei gedrückter Taste prüfen, die Position durch weitere Bewegungen anpassen und die Taste am Ziel loslassen.

    Wählen Sie Felder mit Selektoren oder Referenzen aus strukturierten Ansichten. Geben Sie Text ein, aktivieren oder deaktivieren Sie Kontrollkästchen, wählen Sie Optionsfelder und eine oder mehrere Optionen nativer Listen anhand ihrer Beschriftung. Ersetzen oder löschen Sie Text, oder schreiben Sie an der Cursorposition weiter. Tastenkürzel nutzen den aktuellen Fokus. Der langsame Modus fügt zufällige Pausen zwischen Zeichen und Aktionen ein.

    Warten Sie mit Selektoren oder Referenzen aus strukturierten Ansichten auf Inhalte oder Ladeanzeigen. Sie können auch auf Ladeereignisse, das Ende der Netzwerkaktivität oder eine feste Zeitspanne warten.

  • browser_evaluate: Führt JavaScript auf der aktuellen Seite aus, um eigene Daten zu extrahieren, Berechnungen auszuführen, den Anwendungszustand zu lesen oder Inhalte zu ändern. Unterstützt asynchrone Skripte und gibt deren Ergebnisse als JSON zurück.

  • browser_observe: Hilft dem Agenten, Bewegungen und Änderungen anhand einer kurzen Bildfolge der aktuellen Seite zu verstehen.
  • browser_network_requests: Durchsucht abgeschlossene Browseranfragen über Tabs, Seitenwechsel und Aktionen hinweg. Finden Sie gespeicherte API-Aufrufe, HTTP-Fehler und Weiterleitungen. Statische Ressourcen werden standardmäßig ausgeblendet, um die Ergebnismenge zu verringern. Fehlgeschlagene oder unvollständige Anfragen werden ausgelassen. Gibt strukturierte Daten mit Seiteneinteilung zurück.
  • browser_network_request: Liest die Zusammenfassung, Header, gesendeten Daten oder den Textinhalt einer gespeicherten Anfrage als strukturiertes JSON, ohne die Anfrage zu wiederholen. Gibt den gewählten Teil vollständig zurück. Gespeicherte Antworten bleiben nach Seitenwechseln verfügbar, auch HTTP-Antworten mit 4xx und 5xx. Einträge erscheinen, sobald die Aufzeichnung abgeschlossen ist. Lesen des Verlaufs oder Schließen der Sitzung wartet nicht auf laufende Aufzeichnungen. Diese können bei einem Seitenwechsel oder beim Schließen verloren gehen. Die Bytes stammen von Playwright und können von den vom Server gesendeten Bytes abweichen. Für den Text wird die UTF-8-Dekodierung von Playwright genutzt, wenn sie gültig ist, sonst der angegebene Zeichensatz. Nur Inhalte mit einem unterstützten textbasierten Content-Type werden gelesen und gespeichert. Bei binären, fehlenden oder unbekannten Typen bleiben Metadaten und Header erhalten, ohne den Inhalt zu lesen. Das Laden der Seite ändert sich dadurch nicht. Der Verlauf hat Größenlimits. Ein optionaler Hinweis erklärt, ob der Inhalt kein Text ist, zu groß ist oder nicht gelesen werden kann. Lesen Sie benötigte Daten, bevor Sie die MCP-Sitzung schließen.
  • browser_fetch: Ruft eine Seite in einer offenen Stealth-Browsersitzung ab und extrahiert HTML, Markdown, Text oder eine strukturierte Ansicht. Die Ansicht kann die ganze Seite oder ein Element umfassen und enthält Referenzen, Positionen und Größen für spätere Aktionen.

scrapling.core.ai.server.ScraplingMCPServer

ScraplingMCPServer(executable_path=None, auth_token=None)

Erstellt einen Scrapling-MCP-Server.

PARAMETER BESCHREIBUNG
executable_path

Optionaler globaler Pfad zu einer ausführbaren Browser-Datei, die mit Chromium kompatibel ist, für die Browser-Werkzeuge. Ohne Angabe wird die Umgebungsvariable SCRAPLING_EXECUTABLE_PATH verwendet, falls sie gesetzt ist.

TYP: Optional[str] STANDARDWERT: None

auth_token

Optionales gemeinsames Token, das Clients als Authorization: Bearer <token> senden müssen. Ohne Angabe wird die Umgebungsvariable SCRAPLING_MCP_AUTH_TOKEN verwendet, falls sie gesetzt ist. Gilt nur für den Transport streamable-http.

TYP: Optional[str] STANDARDWERT: None

Quellcode in scrapling/core/ai/server.py
def __init__(self, executable_path: Optional[str] = None, auth_token: Optional[str] = None):
    """Create a Scrapling MCP server.

    :param executable_path: Optional global Chromium-compatible browser executable path for browser tools.
        If omitted, the SCRAPLING_EXECUTABLE_PATH environment variable is used when set.
    :param auth_token: Optional shared token that clients must send as `Authorization: Bearer <token>`.
        If omitted, the SCRAPLING_MCP_AUTH_TOKEN environment variable is used when set. It only applies
        to the streamable-http transport.
    """
    self._sessions: Dict[str, _SessionEntry] = {}
    self._executable_path = executable_path or environ.get(MCP_EXECUTABLE_PATH_ENV) or None
    self._auth_token = auth_token or environ.get(MCP_AUTH_TOKEN_ENV) or None

browser_open async

browser_open(
    session_id=None,
    headless=True,
    real_chrome=False,
    timezone_id=None,
    locale=None,
    useragent=None,
    proxy=None,
    cdp_url=None,
    new_context=True,
    executable_path=None,
    cookies=None,
    hide_canvas=False,
    block_webrtc=False,
    allow_webgl=True,
    additional_args=None,
)

Open a reusable local or remote browser session. Pass per-request options to browser_fetch.

Quellcode in scrapling/core/ai/server.py
async def browser_open(
    self,
    session_id: Annotated[
        Optional[str], Field(description="Custom ID; a random 12-character hex ID if omitted.")
    ] = None,
    headless: Annotated[bool, Field(description="Hide the browser window.")] = True,
    real_chrome: Annotated[bool, Field(description="Use locally installed Chrome.")] = False,
    timezone_id: Annotated[
        str | None, Field(description="Browser timezone; uses the system timezone if omitted.")
    ] = None,
    locale: Annotated[
        str | None,
        Field(description="Browser language, Accept-Language, and formatting locale; system default if omitted."),
    ] = None,
    useragent: Annotated[
        Optional[str],
        Field(description="User-Agent override; otherwise generated in headless mode, native in headful mode."),
    ] = None,
    proxy: Annotated[
        Optional[str | Dict[str, str]],
        Field(description="Proxy URL, or dictionary with server and optional username/password."),
    ] = None,
    cdp_url: Annotated[
        Optional[str],
        Field(
            description="Connect to an existing Chromium browser over CDP; browser launch settings do not apply."
        ),
    ] = None,
    new_context: Annotated[
        bool,
        Field(
            description="With cdp_url, create and close a separate context. False reuses and preserves the default context, ignoring useragent, locale, timezone_id, proxy and additional_args."
        ),
    ] = True,
    executable_path: Annotated[
        Optional[str],
        Field(description="Absolute Chromium-compatible executable path; overrides the server default."),
    ] = None,
    cookies: Annotated[
        Sequence[SetCookieParam] | None, Field(description="Initial cookies as Playwright cookie dictionaries.")
    ] = None,
    hide_canvas: Annotated[bool, Field(description="Add noise to canvas operations.")] = False,
    block_webrtc: Annotated[bool, Field(description="Disable non-proxied WebRTC UDP to reduce IP leaks.")] = False,
    allow_webgl: Annotated[bool, Field(description="Enable WebGL; disabling it can trigger bot detection.")] = True,
    additional_args: Annotated[
        Optional[Dict], Field(description="Browser context options that override Scrapling settings.")
    ] = None,
) -> SessionCreatedModel:
    """Open a reusable local or remote browser session. Pass per-request options to `browser_fetch`."""
    session_id = self._new_session_id(session_id)
    options: Dict[str, Any] = dict(
        proxy=proxy,
        locale=locale,
        cookies=cookies,
        block_ads=True,
        record_requests=True,
        useragent=useragent,
        timezone_id=timezone_id,
        additional_args=additional_args,
    )
    session = (
        AsyncRemoteSession(cdp_url, new_context=new_context, **options)
        if cdp_url is not None
        else AsyncStealthySession(
            headless=headless,
            real_chrome=real_chrome,
            executable_path=self._resolve_executable_path(executable_path),
            hide_canvas=hide_canvas,
            block_webrtc=block_webrtc,
            allow_webgl=allow_webgl,
            **options,
        )
    )
    await session.start()
    return self._register_session(session_id, session, "stealthy")

browser_save_state async

browser_save_state(session_id, path)

Speichert Browser-Cookies, Local Storage und IndexedDB als JSON, das mit Playwright kompatibel ist. Enthält kein sessionStorage.

Quellcode in scrapling/core/ai/server.py
async def browser_save_state(
    self,
    session_id: Annotated[str, Field(description="Browser session ID.")],
    path: Annotated[
        NonEmptyString,
        Field(
            description="File on the MCP server host; relative to its working directory. Overwrites the file; parent must exist."
        ),
    ],
) -> str:
    """Save browser cookies, local storage and IndexedDB as Playwright-compatible JSON. Excludes sessionStorage."""
    await self._get_session(session_id, ["stealthy"]).session.save_state(path)
    return "State saved."

browser_load_state async

browser_load_state(session_id, path)

Ersetzt Browser-Cookies, Local Storage und IndexedDB durch Daten aus einer mit Playwright kompatiblen JSON-Datei.

Lässt sessionStorage und offene Tabs unverändert. Rufen Sie Seiten ab oder laden Sie sie neu, um die wiederhergestellte Anmeldung zu nutzen.

Quellcode in scrapling/core/ai/server.py
async def browser_load_state(
    self,
    session_id: Annotated[str, Field(description="Browser session ID.")],
    path: Annotated[
        NonEmptyString, Field(description="File on the MCP server host; relative to its working directory.")
    ],
) -> str:
    """Replace browser cookies, local storage and IndexedDB from a Playwright-compatible JSON file.

    Leaves sessionStorage and open tabs unchanged. Fetch or reload pages to use the restored login."""
    await self._get_session(session_id, ["stealthy"]).session.load_state(path)
    return "State loaded."

open_request_session async

open_request_session(
    session_id=None, impersonate="chrome", proxy=None
)

Öffnet eine wiederverwendbare HTTP-Sitzung für session_make_request, die Cookies, Verbindungen und Anfrageeinstellungen behält.

Quellcode in scrapling/core/ai/server.py
async def open_request_session(
    self,
    session_id: Annotated[
        Optional[str], Field(description="Custom ID; a random 12-character hex ID if omitted.")
    ] = None,
    impersonate: Annotated[
        _MCPImpersonateType,
        Field(
            description='Browser/version to impersonate; "chrome" uses the latest supported Chrome profile. Lists pick randomly per attempt; None disables impersonation.'
        ),
    ] = "chrome",
    proxy: Annotated[
        Optional[str], Field(description="Proxy URL for all session requests, optionally including credentials.")
    ] = None,
) -> SessionCreatedModel:
    """Open a reusable HTTP session for `session_make_request`, keeping cookies, connections, and request settings."""
    session_id = self._new_session_id(session_id)
    session = FetcherSession(impersonate=impersonate, proxy=proxy)
    await session.__aenter__()
    return self._register_session(session_id, session, "static")

close_session async

close_session(session_id)

Schließt eine Sitzung und gibt ihre Ressourcen frei.

Quellcode in scrapling/core/ai/server.py
async def close_session(
    self,
    session_id: Annotated[str, Field(description="Session ID to close.")],
) -> SessionClosedModel:
    """Close a session and free its resources."""
    entry = self._sessions.pop(session_id, None)
    if entry is None:
        raise ValueError(f"Session '{session_id}' not found. Use list_sessions to see active sessions.")

    if entry.session_type == "static":
        await entry.session.__aexit__(None, None, None)
    else:
        await entry.session.close()
    return SessionClosedModel(
        session_id=session_id,
        message=f"Session '{session_id}' closed successfully.",
    )

list_sessions async

list_sessions()

Listet registrierte Sitzungen, ihren Status und die tatsächlich verwendeten Einstellungen auf.

Quellcode in scrapling/core/ai/server.py
async def list_sessions(self) -> List[SessionInfo]:
    """List registered sessions, their status, and effective settings."""
    return [
        SessionInfo(
            session_id=sid,
            session_type=entry.session_type,
            created_at=entry.created_at,
            is_alive=entry.session._is_alive,
            settings=_session_settings(entry.session),
        )
        for sid, entry in self._sessions.items()
    ]

browser_network_requests async

browser_network_requests(
    session_id,
    url_pattern=None,
    include_static=False,
    after_id=0,
    limit=50,
)

List completed requests across page changes, with IDs and pagination. Recording starts when the browser session opens and closing does not wait for running captures.

Quellcode in scrapling/core/ai/server.py
async def browser_network_requests(
    self,
    session_id: Annotated[str, Field(description="Browser session ID.")],
    url_pattern: Annotated[Optional[str], Field(description="Filter URLs by regular expression.")] = None,
    include_static: Annotated[bool, Field(description="Include successful non-API requests too.")] = False,
    after_id: Annotated[
        int, Field(ge=0, description="Return requests with a higher ID; use next_cursor for more.")
    ] = 0,
    limit: Annotated[int, Field(ge=1, le=200, description="Maximum requests to return.")] = 50,
) -> NetworkRequestsModel:
    """List completed requests across page changes, with IDs and pagination. Recording starts when the browser session opens and closing does not wait for running captures."""
    network = self._get_session(session_id, ["stealthy"]).session.network
    records = network.search(
        url_pattern=url_pattern, include_static=include_static, after_id=after_id, limit=limit + 1
    )
    shown = records[:limit]
    return NetworkRequestsModel(
        requests=[NetworkRequestInfo.model_validate(record, from_attributes=True) for record in shown],
        next_cursor=shown[-1].meta["network_id"] if shown else max(after_id, network.last_id),
        has_more=len(records) > limit,
        dropped_count=network.dropped_count,
    )

browser_network_request async

browser_network_request(
    session_id, request_id, part="summary"
)

Liest einen gespeicherten Anfrageteil vollständig, ohne die Anfrage erneut zu senden. Gibt strukturierte Daten und Hinweise zur Verfügbarkeit zurück.

Quellcode in scrapling/core/ai/server.py
async def browser_network_request(
    self,
    session_id: Annotated[str, Field(description="Browser session ID.")],
    request_id: Annotated[int, Field(ge=1, description="ID from `browser_network_requests`.")],
    part: Annotated[
        NetworkPart, Field(description="Request summary, headers, or text body; incomplete headers are marked.")
    ] = "summary",
) -> NetworkRequestModel:
    """Read a complete saved request part without resending it, with structured data and availability notes."""
    network = self._get_session(session_id, ["stealthy"]).session.network
    record = network.get(request_id)
    if record is None:
        raise ValueError(
            f"Request {request_id} was not found or is no longer retained. Use browser_network_requests."
        )
    return _request_details(record, part)

browser_extract async

browser_extract(
    session_id,
    extraction_type="snapshot",
    target=None,
    depth=None,
    boxes=True,
    search=None,
    main_content_only=True,
)

Liest die aktuelle Seite oder ein Element als strukturierte Ansicht, HTML, Markdown oder Text, ohne die Seite zu wechseln. Gibt Klartext zurück. Die Suche in der Seitenansicht behält Referenzen, Pfade der Vorfahren und drei Zeilen nahen Kontext.

Quellcode in scrapling/core/ai/server.py
async def browser_extract(
    self,
    session_id: Annotated[str, Field(description="Browser session ID")],
    extraction_type: Annotated[
        SessionExtractionType, Field(description="Output format; defaults to an AI ARIA snapshot.")
    ] = "snapshot",
    target: Annotated[
        Optional[NonEmptyString],
        Field(description="Playwright selector or aria-ref=<ref> matching one element; omitted reads the page."),
    ] = None,
    depth: Annotated[
        Optional[int],
        Field(description="Snapshot depth before search; unlimited if omitted. Ignored for other formats."),
    ] = None,
    boxes: Annotated[
        bool, Field(description="Include viewport bounding boxes in snapshots; ignored for other formats.")
    ] = True,
    search: Annotated[
        Optional[NonEmptyString],
        Field(description="Snapshot-only Python regex; case-sensitive unless (?i). No matches returns empty text."),
    ] = None,
    main_content_only: Annotated[
        bool, Field(description="Clean body/target content for AI; ignored for snapshots.")
    ] = True,
) -> str:
    """Read the current page or one element as snapshot, HTML, Markdown, or text without navigating.
    Return plain text; snapshot search keeps refs, ancestor paths and three lines of nearby context."""
    pattern = None
    if search is not None:
        if extraction_type != "snapshot":
            raise ValueError("search requires snapshot output.")
        try:
            pattern = re.compile(search)
        except re.error as error:
            raise ValueError(f"Invalid search regex: {error}") from error
    if extraction_type == "snapshot":
        snapshot = await self._browser_snapshot(session_id, css_selector=target, depth=depth, boxes=boxes)
        return _search_snapshot(snapshot, pattern) if pattern is not None else snapshot
    with self._browser_page(session_id) as (_, page):
        content = (
            await page.locator(target).evaluate("(element) => element.outerHTML")
            if target is not None
            else await ResponseFactory._get_async_page_content(page)
        )
        return _CONTROL_CHARS_PATTERN.sub(
            "",
            "".join(
                Convertor._extract_content(
                    Selector(content=content, url=page.url),
                    extraction_type=extraction_type,
                    main_content_only=main_content_only,
                )
            ),
        )

browser_actions async

browser_actions(session_id, actions, slowly=False)

Chain all kinds of browser actions. Do actions without multiple calls or plan ahead to execute a sequence of actions. Run mouse, field, key, dialog, and wait actions on the current page in order. Load waits observe the current document; wait for a result element for delayed navigation. Stops on the first error, identifying its action; partial effects remain. No retries. Element mouse targets auto-wait and scroll into view; coordinates and wheel do not wait for navigation or scrolling. mouse_down holds across calls for observation and further moves; mouse_up releases, including after errors. For drag and drop, move over the target twice before releasing. Cancelled/timed-out clicks attempt release.

Quellcode in scrapling/core/ai/server.py
async def browser_actions(
    self,
    session_id: Annotated[str, Field(description="Browser session ID")],
    actions: Annotated[
        List[BrowserAction],
        Field(
            min_length=1,
            description="Ordered actions; fields need target; move/click need target or viewport (x, y); mouse_down/mouse_up act at the current pointer.",
        ),
    ],
    slowly: Annotated[
        bool,
        Field(
            description="Fresh random delays: 50-150 ms per character, 100-300 ms between all actions, including waits."
        ),
    ] = False,
) -> str:
    """Chain all kinds of browser actions. Do actions without multiple calls or plan ahead to execute a sequence of actions.
    Run mouse, field, key, dialog, and wait actions on the current page in order.
    Load waits observe the current document; wait for a result element for delayed navigation.
    Stops on the first error, identifying its action; partial effects remain. No retries.
    Element mouse targets auto-wait and scroll into view; coordinates and wheel do not wait for navigation or scrolling.
    mouse_down holds across calls for observation and further moves; mouse_up releases, including after errors.
    For drag and drop, move over the target twice before releasing. Cancelled/timed-out clicks attempt release."""
    _validate_actions(actions)
    with self._browser_page(session_id) as (_, page):
        await _run_actions(page, actions, slowly)
    return "Actions completed."

browser_evaluate async

browser_evaluate(
    session_id, expression, arg=None, isolated_context=True
)

Führt JavaScript auf der aktuellen Seite aus, wartet auf Promises und gibt JSON-Text ohne Seitenansicht zurück. Geben Sie JSON-kompatible Werte zurück. null/undefined werden zu null. Änderungen durch das Skript werden bei Fehlern nicht rückgängig gemacht. Ein Abbruch beendet das Warten, aber möglicherweise nicht das Skript.

Quellcode in scrapling/core/ai/server.py
async def browser_evaluate(
    self,
    session_id: Annotated[str, Field(description="Browser session ID")],
    expression: Annotated[NonEmptyString, Field(description="JavaScript expression or function to invoke.")],
    arg: Annotated[
        Optional[Dict[str, Any]],
        Field(description="JSON object passed to the function; use properties for scalar or array values."),
    ] = None,
    isolated_context: Annotated[
        bool,
        Field(
            description="False accesses the page's own JS variables; True uses a separate context with the same DOM."
        ),
    ] = True,
) -> str:
    """Run JavaScript on the current page, await promises, and return JSON text without a snapshot.
    Return JSON-compatible values; null/undefined become null. Script effects are not undone on failure.
    Cancellation stops waiting, but may not stop the script."""
    with self._browser_page(session_id) as (_, page):
        return dumps(
            await page.evaluate(expression, arg=arg, isolated_context=isolated_context),
            ensure_ascii=False,
            allow_nan=False,
            separators=(",", ":"),
        )

browser_screenshot async

browser_screenshot(
    session_id,
    image_type="png",
    full_page=False,
    quality=None,
    timeout=30000,
    selector=None,
    ref=None,
)

Erfasst die aktuelle Seite oder ein Element, ohne die Seite zu wechseln. Gibt das Bild und die aktuelle URL zurück. Bei der Aufnahme eines Elements wird gewartet und ins Blickfeld gescrollt. Verwenden Sie browser_actions für weitere Wartebedingungen vor der Aufnahme.

Quellcode in scrapling/core/ai/server.py
async def browser_screenshot(
    self,
    session_id: Annotated[str, Field(description="Browser session ID")],
    image_type: Annotated[ScreenshotType, Field(description="Image format.")] = "png",
    full_page: Annotated[
        bool, Field(description="Capture the full scrollable page; incompatible with selector/ref.")
    ] = False,
    quality: Annotated[
        Optional[Annotated[int, Field(ge=0, le=100)]], Field(description="JPEG quality, 0-100; invalid for PNG.")
    ] = None,
    timeout: Annotated[
        NonNegativeFiniteFloat, Field(description="Capture limit in milliseconds; 0 disables it.")
    ] = 30000,
    selector: Annotated[
        Optional[NonEmptyString],
        Field(description="Playwright selector matching one element; omit when using ref."),
    ] = None,
    ref: Annotated[
        Optional[NonEmptyString],
        Field(description="Current snapshot element reference; omit both targets for page capture."),
    ] = None,
) -> List[ImageContent | TextContent]:
    """Capture the current page or one element without navigating; return the image and current URL.
    Element capture waits and scrolls into view. Use browser_actions for other waits before capture."""
    if quality is not None and image_type != "jpeg":
        raise ValueError("'quality' is only valid when 'image_type' is 'jpeg'.")
    if selector is not None and ref is not None:
        raise ValueError("Provide either 'selector' or 'ref', not both.")
    if full_page and (selector is not None or ref is not None):
        raise ValueError("'full_page' cannot be combined with 'selector' or 'ref'.")

    with self._browser_page(session_id) as (_, page):
        options: Dict[str, Any] = {"type": image_type, "quality": quality, "timeout": timeout}
        if selector is not None or ref is not None:
            started = monotonic()
            element = await page.locator(selector or f"aria-ref={ref}").element_handle(timeout=timeout)
            try:
                options["timeout"] = max(1, timeout - (monotonic() - started) * 1000) if timeout else 0
                data = await element.screenshot(**options)
            except BaseException as exc:
                try:
                    with CancelScope(shield=True):
                        await element.dispose()
                except Exception as cleanup_error:
                    raise exc from cleanup_error
                raise
            with CancelScope(shield=True):
                await element.dispose()
        else:
            data = await page.screenshot(full_page=full_page, **options)
        return [Image(data=data, format=image_type).to_image_content(), TextContent(type="text", text=page.url)]

browser_observe async

browser_observe(
    session_id, duration_ms=1500, max_frames=12
)

Inspect browser page motion through a short sequence of timestamped viewport images. The page keeps running during and after capture. Use this tool when you get any challenge that require timed actions like games, appearing/disappearing CAPTCHAs, or interacting with any moving elements.

Quellcode in scrapling/core/ai/server.py
async def browser_observe(
    self,
    session_id: Annotated[str, Field(description="Browser session ID")],
    duration_ms: Annotated[int, Field(ge=100, le=5000, description="Observation window in milliseconds.")] = 1500,
    max_frames: Annotated[int, Field(ge=2, le=20, description="Maximum number of sampled frames.")] = 12,
) -> List[ImageContent | TextContent]:
    """Inspect browser page motion through a short sequence of timestamped viewport images. The page keeps running during and after capture.
    Use this tool when you get any challenge that require timed actions like games, appearing/disappearing CAPTCHAs, or interacting with any moving elements.
    """
    with self._browser_page(session_id) as (_, page):
        frames: Dict[int, Dict[str, Any]] = {}
        started = monotonic()

        def capture(frame: Dict[str, Any]) -> None:
            elapsed = (monotonic() - started) * 1000
            if elapsed < duration_ms:
                frames[int(elapsed * max_frames / duration_ms)] = frame

        try:
            await page.screencast.start(on_frame=capture, quality=60, size={"width": 800, "height": 800})
            await sleep(max(0, duration_ms / 1000 - (monotonic() - started)))
        except BaseException as error:
            try:
                await _stop_screencast(page)
            except Exception as cleanup_error:
                raise error from cleanup_error
            raise
        await _stop_screencast(page)
        if not frames:
            raise RuntimeError("No frames were captured. Try browser_screenshot or a longer observation window.")
        return [
            content
            for frame in frames.values()
            for content in (
                TextContent(
                    type="text",
                    text=dumps(
                        {
                            "timestamp_ms": frame["timestamp"],
                            "viewport_width": frame["viewportWidth"],
                            "viewport_height": frame["viewportHeight"],
                        },
                        separators=(",", ":"),
                    ),
                ),
                Image(data=frame["data"], format="jpeg").to_image_content(),
            )
        ]

make_request async staticmethod

make_request(
    url,
    method="GET",
    impersonate="chrome",
    extraction_type="markdown",
    css_selector=None,
    main_content_only=True,
    params=None,
    data=None,
    json=None,
    headers=None,
    cookies=None,
    timeout=30,
    follow_redirects="safe",
    max_redirects=30,
    retries=3,
    retry_delay=1,
    proxy=None,
    proxy_auth=None,
    auth=None,
    verify=True,
    http3=False,
    stealthy_headers=True,
)

Sendet eine HTTP-Anfrage ohne JavaScript. Geeignet für geringen bis mittleren Schutz.

Quellcode in scrapling/core/ai/server.py
@staticmethod
async def make_request(
    url: Annotated[str, Field(description="URL to fetch.")],
    method: Annotated[SUPPORTED_HTTP_METHODS, Field(description="HTTP method.")] = "GET",
    impersonate: Annotated[
        _MCPImpersonateType,
        Field(
            description='Browser/version to impersonate; "chrome" uses the latest supported Chrome profile. Lists pick randomly per attempt; None disables impersonation.'
        ),
    ] = "chrome",
    extraction_type: Annotated[extraction_types, Field(description="Content output format.")] = "markdown",
    css_selector: Annotated[
        Optional[str], Field(description="Select matching elements after `main_content_only` filtering.")
    ] = None,
    main_content_only: Annotated[
        bool, Field(description="Sanitize <body> content before selection; False uses the full document.")
    ] = True,
    params: Annotated[Optional[Dict], Field(description="Query parameters.")] = None,
    data: Annotated[Optional[Dict[str, str] | str], Field(description="Form or raw body; ignored for GET.")] = None,
    json: Annotated[Optional[Dict | List], Field(description="JSON body; ignored for GET.")] = None,
    headers: Annotated[Optional[Mapping[str, Optional[str]]], Field(description="Request headers.")] = None,
    cookies: Annotated[Optional[Dict[str, str]], Field(description="Request cookies.")] = None,
    timeout: Annotated[Optional[int | float], Field(description="Timeout in seconds.")] = 30,
    follow_redirects: Annotated[
        FollowRedirects,
        Field(
            description='"safe" blocks redirects to private/internal IPs; True allows all; False disables redirects.'
        ),
    ] = "safe",
    max_redirects: Annotated[int, Field(description="Redirect limit; -1 means unlimited.")] = 30,
    retries: Annotated[
        Optional[int], Field(description="Maximum attempts, including the first; None or values below 1 send once.")
    ] = 3,
    retry_delay: Annotated[Optional[int], Field(description="Seconds between retries.")] = 1,
    proxy: Annotated[Optional[str], Field(description="Proxy URL, optionally including credentials.")] = None,
    proxy_auth: Annotated[
        Optional[Dict[str, str]], Field(description='Proxy basic auth: {"username": ..., "password": ...}.')
    ] = None,
    auth: Annotated[
        Optional[Dict[str, str]], Field(description='Basic auth: {"username": ..., "password": ...}.')
    ] = None,
    verify: Annotated[Optional[bool], Field(description="Verify TLS certificates.")] = True,
    http3: Annotated[
        Optional[bool], Field(description="Use HTTP/3; may conflict with browser impersonation.")
    ] = False,
    stealthy_headers: Annotated[
        Optional[bool],
        Field(description="Fill missing browser headers and use a Google referer if none was supplied."),
    ] = True,
) -> ResponseModel:
    """Make an HTTP request without JavaScript. Suitable for low-to-mid protection."""
    normalized_proxy_auth = _normalize_credentials(proxy_auth)
    normalized_auth = _normalize_credentials(auth)

    request_kwargs: Dict[str, Any] = dict(
        auth=normalized_auth,
        proxy=proxy,
        http3=http3,
        verify=verify,
        params=params,
        headers=headers,
        cookies=cookies,
        timeout=timeout,
        retries=retries,
        proxy_auth=normalized_proxy_auth,
        retry_delay=retry_delay,
        impersonate=impersonate,
        max_redirects=max_redirects,
        follow_redirects=follow_redirects,
        stealthy_headers=stealthy_headers,
    )
    if method != "GET":
        request_kwargs.update(data=data, json=json)

    async with FetcherSession() as session:
        page = await getattr(session, method.lower())(url, **request_kwargs)
        return _translate_response(page, extraction_type, css_selector, main_content_only)

browser_fetch_once async

browser_fetch_once(
    url,
    extraction_type="markdown",
    css_selector=None,
    main_content_only=True,
    headless=True,
    google_search=True,
    real_chrome=False,
    wait=0,
    proxy=None,
    timezone_id=None,
    locale=None,
    extra_headers=None,
    useragent=None,
    hide_canvas=False,
    executable_path=None,
    timeout=30000,
    disable_resources=False,
    wait_selector=None,
    cookies=None,
    network_idle=False,
    wait_selector_state="attached",
    block_webrtc=False,
    allow_webgl=True,
    solve_cloudflare=False,
    additional_args=None,
    pierce_shadow=False,
)

Ruft eine Seite mit einem Stealthy-Browser und JavaScript-Ausführung ab.

Quellcode in scrapling/core/ai/server.py
async def browser_fetch_once(
    self,
    url: Annotated[str, Field(description="URL to fetch.")],
    extraction_type: Annotated[extraction_types, Field(description="Content output format.")] = "markdown",
    css_selector: Annotated[
        Optional[str], Field(description="Select matching elements after `main_content_only` filtering.")
    ] = None,
    main_content_only: Annotated[
        bool, Field(description="Sanitize <body> content before selection; False uses the full document.")
    ] = True,
    headless: Annotated[bool, Field(description="Hide the browser window.")] = True,
    google_search: Annotated[
        bool, Field(description="Set a Google referer, overriding any supplied referer.")
    ] = True,
    real_chrome: Annotated[bool, Field(description="Use locally installed Chrome.")] = False,
    wait: Annotated[
        int | float, Field(description="Extra milliseconds after the page is ready, before returning.")
    ] = 0,
    proxy: Annotated[
        Optional[str | Dict[str, str]],
        Field(description="Proxy URL, or dictionary with server and optional username/password."),
    ] = None,
    timezone_id: Annotated[
        str | None, Field(description="Browser timezone; uses the system timezone if omitted.")
    ] = None,
    locale: Annotated[
        str | None,
        Field(description="Browser language, Accept-Language, and formatting locale; system default if omitted."),
    ] = None,
    extra_headers: Annotated[Optional[Dict[str, str]], Field(description="Additional request headers.")] = None,
    useragent: Annotated[
        Optional[str],
        Field(description="User-Agent override; otherwise generated in headless mode, native in headful mode."),
    ] = None,
    hide_canvas: Annotated[bool, Field(description="Add noise to canvas operations.")] = False,
    executable_path: Annotated[
        Optional[str],
        Field(description="Absolute Chromium-compatible executable path; overrides the server default."),
    ] = None,
    timeout: Annotated[
        int | float, Field(description="Navigation and page-operation timeout in milliseconds.")
    ] = 30000,
    disable_resources: Annotated[
        bool,
        Field(
            description="Block font, image, media, beacon, object, imageset, texttrack, websocket, csp_report, and stylesheet requests."
        ),
    ] = False,
    wait_selector: Annotated[
        Optional[str], Field(description="Wait for the first CSS match; continue if the wait fails.")
    ] = None,
    cookies: Annotated[
        Sequence[SetCookieParam] | None, Field(description="Initial cookies as Playwright cookie dictionaries.")
    ] = None,
    network_idle: Annotated[
        bool, Field(description="Try to wait for 500 ms without network activity; continue if the wait fails.")
    ] = False,
    wait_selector_state: Annotated[
        SelectorWaitStates, Field(description="Target state of `wait_selector`.")
    ] = "attached",
    block_webrtc: Annotated[bool, Field(description="Disable non-proxied WebRTC UDP to reduce IP leaks.")] = False,
    allow_webgl: Annotated[bool, Field(description="Enable WebGL; disabling it can trigger bot detection.")] = True,
    solve_cloudflare: Annotated[
        bool, Field(description="Attempt to solve Cloudflare Turnstile/interstitial challenges before returning.")
    ] = False,
    additional_args: Annotated[
        Optional[Dict], Field(description="Browser context options that override Scrapling settings.")
    ] = None,
    pierce_shadow: Annotated[bool, Field(description="Include open Shadow DOM content.")] = False,
) -> ResponseModel:
    """Fetch a page with a stealth browser and JavaScript rendering."""
    async with AsyncStealthySession(
        wait=wait,
        proxy=proxy,
        locale=locale,
        timeout=timeout,
        cookies=cookies,
        headless=headless,
        block_ads=True,
        max_pages=1,
        useragent=useragent,
        timezone_id=timezone_id,
        real_chrome=real_chrome,
        hide_canvas=hide_canvas,
        allow_webgl=allow_webgl,
        network_idle=network_idle,
        pierce_shadow=pierce_shadow,
        block_webrtc=block_webrtc,
        wait_selector=wait_selector,
        google_search=google_search,
        extra_headers=extra_headers,
        executable_path=self._resolve_executable_path(executable_path),
        additional_args=additional_args,
        solve_cloudflare=solve_cloudflare,
        disable_resources=disable_resources,
        wait_selector_state=wait_selector_state,
    ) as session:
        page = await session.fetch(url)

    return _translate_response(page, extraction_type, css_selector, main_content_only)

browser_fetch async

browser_fetch(
    url,
    session_id,
    extraction_type="markdown",
    css_selector=None,
    main_content_only=True,
    wait=0,
    timeout=30000,
    google_search=True,
    network_idle=False,
    load_dom=True,
    disable_resources=False,
    wait_selector=None,
    wait_selector_state="attached",
    extra_headers=None,
    blocked_domains=None,
    solve_cloudflare=False,
    pierce_shadow=False,
)

Ruft eine URL in einer offenen Stealthy-Browser-Sitzung ab. Optionen gelten nur für diese Anfrage.

Quellcode in scrapling/core/ai/server.py
async def browser_fetch(
    self,
    url: Annotated[str, Field(description="URL to fetch.")],
    session_id: Annotated[str, Field(description="Browser session ID.")],
    extraction_type: Annotated[
        SessionExtractionType, Field(description="Content output format; snapshots include element bounding boxes.")
    ] = "markdown",
    css_selector: Annotated[
        Optional[str],
        Field(description="Select after `main_content_only` filtering; snapshots require exactly one match."),
    ] = None,
    main_content_only: Annotated[
        bool,
        Field(description="Sanitize <body> before selection; False uses the full document. Ignored for snapshots."),
    ] = True,
    wait: Annotated[
        int | float, Field(description="Extra milliseconds after the page is ready, before returning.")
    ] = 0,
    timeout: Annotated[
        int | float, Field(description="Navigation and page-operation timeout in milliseconds.")
    ] = 30000,
    google_search: Annotated[
        bool, Field(description="Set a Google referer, overriding any supplied referer.")
    ] = True,
    network_idle: Annotated[
        bool, Field(description="Try to wait for 500 ms without network activity; continue if the wait fails.")
    ] = False,
    load_dom: Annotated[bool, Field(description="Wait for DOMContentLoaded.")] = True,
    disable_resources: Annotated[
        bool,
        Field(
            description="Block font, image, media, beacon, object, imageset, texttrack, websocket, csp_report, and stylesheet requests."
        ),
    ] = False,
    wait_selector: Annotated[
        Optional[str], Field(description="Wait for the first CSS match; continue if the wait fails.")
    ] = None,
    wait_selector_state: Annotated[
        SelectorWaitStates, Field(description="Target state of `wait_selector`.")
    ] = "attached",
    extra_headers: Annotated[Optional[Dict[str, str]], Field(description="Additional request headers.")] = None,
    blocked_domains: Annotated[
        Optional[Set[str]], Field(description="Block requests to these domains and their subdomains.")
    ] = None,
    solve_cloudflare: Annotated[
        bool, Field(description="Attempt to solve Cloudflare Turnstile/interstitial challenges.")
    ] = False,
    pierce_shadow: Annotated[
        bool, Field(description="Include open Shadow DOM content; ignored for snapshots.")
    ] = False,
) -> ResponseModel:
    """Fetch a URL in an open stealthy browser session. Options apply only to this request."""
    entry = self._get_session(session_id, expected_type=["stealthy"])
    fetch_params = dict(
        wait=wait,
        timeout=timeout,
        google_search=google_search,
        network_idle=network_idle,
        pierce_shadow=pierce_shadow,
        load_dom=load_dom,
        disable_resources=disable_resources,
        wait_selector=wait_selector,
        wait_selector_state=wait_selector_state,
        extra_headers=extra_headers,
        blocked_domains=blocked_domains,
        solve_cloudflare=solve_cloudflare,
    )
    page = await entry.session.fetch(
        url, **{name: value for name, value in fetch_params.items() if name in _STEALTH_FETCH_KEYS}
    )
    if extraction_type == "snapshot":
        return ResponseModel(
            status=page.status, content=[await self._browser_snapshot(session_id, css_selector)], url=page.url
        )
    return _translate_response(page, extraction_type, css_selector, main_content_only)

session_make_request async

session_make_request(
    url,
    session_id,
    method="GET",
    extraction_type="markdown",
    css_selector=None,
    main_content_only=True,
    params=None,
    data=None,
    json=None,
    headers=None,
    cookies=None,
    timeout=30,
    follow_redirects="safe",
    max_redirects=30,
    retries=3,
    retry_delay=1,
    auth=None,
    verify=True,
    http3=False,
    stealthy_headers=True,
)

Sendet eine HTTP-Anfrage mit den Cookies, Verbindungen, der Browser-Nachbildung und dem Proxy der Sitzung. Optionen gelten für diese Anfrage.

Quellcode in scrapling/core/ai/server.py
async def session_make_request(
    self,
    url: Annotated[str, Field(description="URL to fetch.")],
    session_id: Annotated[str, Field(description="HTTP session ID from `open_request_session`.")],
    method: Annotated[SUPPORTED_HTTP_METHODS, Field(description="HTTP method.")] = "GET",
    extraction_type: Annotated[extraction_types, Field(description="Content output format.")] = "markdown",
    css_selector: Annotated[
        Optional[str], Field(description="Select matching elements after `main_content_only` filtering.")
    ] = None,
    main_content_only: Annotated[
        bool, Field(description="Sanitize <body> content before selection; False uses the full document.")
    ] = True,
    params: Annotated[Optional[Dict], Field(description="Query parameters.")] = None,
    data: Annotated[Optional[Dict[str, str] | str], Field(description="Form or raw body; ignored for GET.")] = None,
    json: Annotated[Optional[Dict | List], Field(description="JSON body; ignored for GET.")] = None,
    headers: Annotated[Optional[Mapping[str, Optional[str]]], Field(description="Request headers.")] = None,
    cookies: Annotated[Optional[Dict[str, str]], Field(description="Request cookies.")] = None,
    timeout: Annotated[Optional[int | float], Field(description="Timeout in seconds.")] = 30,
    follow_redirects: Annotated[
        FollowRedirects,
        Field(
            description='"safe" blocks redirects to private/internal IPs; True allows all; False disables redirects.'
        ),
    ] = "safe",
    max_redirects: Annotated[int, Field(description="Redirect limit; -1 means unlimited.")] = 30,
    retries: Annotated[
        Optional[int], Field(description="Maximum attempts, including the first; None or values below 1 send once.")
    ] = 3,
    retry_delay: Annotated[Optional[int], Field(description="Seconds between retries.")] = 1,
    auth: Annotated[
        Optional[Dict[str, str]], Field(description='Basic auth: {"username": ..., "password": ...}.')
    ] = None,
    verify: Annotated[Optional[bool], Field(description="Verify TLS certificates.")] = True,
    http3: Annotated[
        Optional[bool], Field(description="Use HTTP/3; may conflict with browser impersonation.")
    ] = False,
    stealthy_headers: Annotated[
        Optional[bool],
        Field(description="Fill missing browser headers and use a Google referer if none was supplied."),
    ] = True,
) -> ResponseModel:
    """Make an HTTP request using the session's cookies, connections, impersonation, and proxy. Options apply to this request."""
    entry = self._get_session(session_id, expected_type=["static"])

    request_kwargs: Dict[str, Any] = dict(
        auth=_normalize_credentials(auth),
        http3=http3,
        verify=verify,
        params=params,
        headers=headers,
        cookies=cookies,
        timeout=timeout,
        retries=retries,
        retry_delay=retry_delay,
        max_redirects=max_redirects,
        follow_redirects=follow_redirects,
        stealthy_headers=stealthy_headers,
    )
    if method != "GET":
        request_kwargs.update(data=data, json=json)

    page = await getattr(entry.session._client, method.lower())(url, **request_kwargs)
    return _translate_response(page, extraction_type, css_selector, main_content_only)

serve

serve(
    http,
    host,
    port,
    allowed_hosts=(),
    allow_unauthenticated=False,
)

Startet den MCP-Server.

PARAMETER BESCHREIBUNG
http

Verwendet den Transport streamable-http statt stdio.

TYP: bool

host

Die Adresse, auf der der Server lauscht, wenn http aktiviert ist.

TYP: str

port

Der Port, auf dem der Server lauscht, wenn http aktiviert ist.

TYP: int

allowed_hosts

Erlaubte Hostnamen. Aktiviert den Schutz vor DNS-Rebinding.

TYP: Sequence[str] STANDARDWERT: ()

allow_unauthenticated

Startet den Transport streamable-http ohne Token. Er erfordert standardmäßig eine Authentifizierung. Diese Option schaltet sie ausdrücklich aus.

TYP: bool STANDARDWERT: False

Quellcode in scrapling/core/ai/server.py
def serve(
    self,
    http: bool,
    host: str,
    port: int,
    allowed_hosts: Sequence[str] = (),
    allow_unauthenticated: bool = False,
):
    """Serve the MCP server.

    :param http: Serve over the streamable-http transport instead of stdio.
    :param host: The host to bind to when `http` is enabled.
    :param port: The port to bind to when `http` is enabled.
    :param allowed_hosts: Host names to accept, which turns on DNS-rebinding protection.
    :param allow_unauthenticated: Start the streamable-http transport without a token. The transport
        requires authentication by default, so this is the explicit opt-out.
    """
    if not http:
        if self._auth_token:
            log.warning(
                "The authentication token only applies to the streamable-http transport, so it's ignored with stdio."
            )
    elif self._auth_token:
        if allow_unauthenticated:
            log.warning(
                "An authentication token was given, so it takes precedence and the server still requires it."
            )
    elif not allow_unauthenticated:
        raise ValueError(
            f"Refusing to serve the MCP server over HTTP without authentication because anyone who can reach "
            f"{host}:{port} would be able to use every tool, including fetching arbitrary URLs from this machine. "
            f"Pass `--auth-token` (or set the {MCP_AUTH_TOKEN_ENV} environment variable) to require a bearer "
            f"token, or `--no-auth` to serve it unauthenticated anyway."
        )
    else:
        log.warning(
            f"The MCP server is running over HTTP without authentication, so anyone who can reach "
            f"{host}:{port} can use every tool, including fetching arbitrary URLs from this machine."
        )

    server = self._build_server(host, port)
    if http:
        server.run(
            transport="streamable-http",
            host=host,
            port=port,
            transport_security=self._transport_security(allowed_hosts),
        )
    else:
        server.run()