Interface ScraperOptions

interface ScraperOptions {
    experimental: {
        browserProfile?: Partial<BrowserProfile>;
        flowStepDelay?: number;
        xClientTransactionId: boolean;
        xpff: boolean;
    };
    fetch: {
        (input: RequestInfo | URL, init?: RequestInit): Promise<Response>;
        (input: string | Request | URL, init?: RequestInit): Promise<Response>;
    };
    rateLimitStrategy: RateLimitStrategy;
    transform: Partial<FetchTransformOptions>;
}

Properties

experimental fetch rateLimitStrategy transform

Properties

experimental

experimental: {
    browserProfile?: Partial<BrowserProfile>;
    flowStepDelay?: number;
    xClientTransactionId: boolean;
    xpff: boolean;
}

Experimental features that may be added, changed, or removed at any time. Use with caution.

Type declaration

  • OptionalbrowserProfile?: Partial<BrowserProfile>

    Override the browser profile used for Castle.io fingerprint token generation. Unspecified fields are randomized from realistic value pools. Set this if you want a consistent fingerprint or need to match specific hardware.

  • OptionalflowStepDelay?: number

    Delay in milliseconds between login flow steps, to mimic human-like timing. Without a delay, Twitter may flag rapid-fire requests as bot activity (error 399). Set to 0 to disable. Default is 1-3 seconds (average ~2s) with random jitter.

  • xClientTransactionId: boolean

    Enables the generation of the x-client-transaction-id header on requests. This may resolve some errors.

  • xpff: boolean

    Enables the generation of the x-xp-forwarded-for header on requests. This may resolve some errors.

fetch

fetch: {
    (input: RequestInfo | URL, init?: RequestInit): Promise<Response>;
    (input: string | Request | URL, init?: RequestInit): Promise<Response>;
}

An alternative fetch function to use instead of the default fetch function. This may be useful in nonstandard runtime environments, such as edge workers.

Type declaration

    • (input: RequestInfo | URL, init?: RequestInit): Promise<Response>

    • MDN Reference

      Parameters

      • input: RequestInfo | URL
      • Optionalinit: RequestInit

      Returns Promise<Response>

    • (input: string | Request | URL, init?: RequestInit): Promise<Response>

    • MDN Reference

      Parameters

      • input: string | Request | URL
      • Optionalinit: RequestInit

      Returns Promise<Response>

rateLimitStrategy

rateLimitStrategy: RateLimitStrategy

A handling strategy for rate limits (HTTP 429).

transform

transform: Partial<FetchTransformOptions>

Additional options that control how requests and responses are processed. This can be used to proxy requests through other hosts, for example.

Settings

Member Visibility

On This Page

Properties