Interface ScraperOptions

interface ScraperOptions {
    experimental: { xClientTransactionId: boolean; xpff: boolean };
    fetch: {
        (input: RequestInfo | URL, init?: RequestInit): Promise<Response>;
        (input: string | Request | URL, init?: RequestInit): Promise<Response>;
    };
    rateLimitStrategy: RateLimitStrategy;
    transform: Partial<FetchTransformOptions>;
}

Properties

experimental fetch rateLimitStrategy transform

Properties

experimental

experimental: { xClientTransactionId: boolean; xpff: boolean }

Experimental features that may be added, changed, or removed at any time. Use with caution.

Type declaration

  • xClientTransactionId: boolean

    Enables the generation of the x-client-transaction-id header on requests. This may resolve some errors.

  • xpff: boolean

    Enables the generation of the x-xp-forwarded-for header on requests. This may resolve some errors.

fetch

fetch: {
    (input: RequestInfo | URL, init?: RequestInit): Promise<Response>;
    (input: string | Request | URL, init?: RequestInit): Promise<Response>;
}

An alternative fetch function to use instead of the default fetch function. This may be useful in nonstandard runtime environments, such as edge workers.

Type declaration

    • (input: RequestInfo | URL, init?: RequestInit): Promise<Response>

    • MDN Reference

      Parameters

      • input: RequestInfo | URL
      • Optionalinit: RequestInit

      Returns Promise<Response>

    • (input: string | Request | URL, init?: RequestInit): Promise<Response>

    • MDN Reference

      Parameters

      • input: string | Request | URL
      • Optionalinit: RequestInit

      Returns Promise<Response>

rateLimitStrategy

rateLimitStrategy: RateLimitStrategy

A handling strategy for rate limits (HTTP 429).

transform

transform: Partial<FetchTransformOptions>

Additional options that control how requests and responses are processed. This can be used to proxy requests through other hosts, for example.

Settings

Member Visibility

On This Page

Properties