Class Scraper

An interface to Twitter's undocumented API.

  • Reusing Scraper objects is recommended to minimize the time spent authenticating unnecessarily.

Hierarchy

  • Scraper

Constructors

constructor

Properties

auth authTrends options? token

Methods

clearCookies fetchListTweets fetchProfileFollowers fetchProfileFollowing fetchSearchProfiles fetchSearchTweets getAuthOptions getCookies getFollowers getFollowing getLatestTweet getLikedTweets getProfile getTrends getTweet getTweetWhere getTweets getTweetsAndReplies getTweetsAndRepliesByUserId getTweetsByUserId getTweetsWhere getUserIdByScreenName handleResponse hasGuestToken isLoggedIn login logout searchProfiles searchTweets setCookies withCookie withXCsrfToken

Constructors

constructor

  • Creates a new Scraper object.

    • Scrapers maintain their own guest tokens for Twitter's internal API.
    • Reusing Scraper objects is recommended to minimize the time spent authenticating unnecessarily.

    Parameters

    Returns Scraper

Properties

Private auth

auth: TwitterAuth

Private authTrends

authTrends: TwitterAuth

Private Optional Readonly options

options?: Partial<ScraperOptions>

Private token

token: string

Methods

clearCookies

  • Clear all cookies for the current session.

    Returns Promise<void>

fetchListTweets

  • Fetches list tweets from Twitter.

    Parameters

    • listId: string

      The list id

    • maxTweets: number

      The maximum number of tweets to return.

    • Optional cursor: string

      The search cursor, which can be passed into further requests for more results.

    Returns Promise<QueryTweetsResponse>

    A page of results, containing a cursor that can be used in further requests.

fetchProfileFollowers

  • Fetches profile followers from Twitter.

    Parameters

    • userId: string

      The user whose following should be returned

    • maxProfiles: number

      The maximum number of profiles to return.

    • Optional cursor: string

      The search cursor, which can be passed into further requests for more results.

    Returns Promise<QueryProfilesResponse>

    A page of results, containing a cursor that can be used in further requests.

fetchProfileFollowing

  • Fetches following profiles from Twitter.

    Parameters

    • userId: string

      The user whose following should be returned

    • maxProfiles: number

      The maximum number of profiles to return.

    • Optional cursor: string

      The search cursor, which can be passed into further requests for more results.

    Returns Promise<QueryProfilesResponse>

    A page of results, containing a cursor that can be used in further requests.

fetchSearchProfiles

  • Fetches profiles from Twitter.

    Parameters

    • query: string

      The search query. Any Twitter-compatible query format can be used.

    • maxProfiles: number

      The maximum number of profiles to return.

    • Optional cursor: string

      The search cursor, which can be passed into further requests for more results.

    Returns Promise<QueryProfilesResponse>

    A page of results, containing a cursor that can be used in further requests.

fetchSearchTweets

  • Fetches tweets from Twitter.

    Parameters

    • query: string

      The search query. Any Twitter-compatible query format can be used.

    • maxTweets: number

      The maximum number of tweets to return.

    • searchMode: SearchMode

      The category filter to apply to the search. Defaults to Top.

    • Optional cursor: string

      The search cursor, which can be passed into further requests for more results.

    Returns Promise<QueryTweetsResponse>

    A page of results, containing a cursor that can be used in further requests.

Private getAuthOptions

getCookies

  • Retrieves all cookies for the current session.

    Returns Promise<Cookie[]>

    All cookies for the current session.

getFollowers

  • Fetch the profiles that follow a user

    Parameters

    • userId: string

      The user whose followers should be returned

    • maxProfiles: number

      The maximum number of profiles to return.

    Returns AsyncGenerator<Profile, void, unknown>

    An AsyncGenerator of profiles following the provided user.

getFollowing

  • Fetch the profiles a user is following

    Parameters

    • userId: string

      The user whose following should be returned

    • maxProfiles: number

      The maximum number of profiles to return.

    Returns AsyncGenerator<Profile, void, unknown>

    An AsyncGenerator of following profiles for the provided user.

getLatestTweet

  • Fetches the most recent tweet from a Twitter user.

    Parameters

    • user: string

      The user whose latest tweet should be returned.

    • includeRetweets: boolean = false

      Whether or not to include retweets. Defaults to false.

    • max: number = 200

    Returns Promise<null | void | Tweet>

    The Tweet object or null/undefined if it couldn't be fetched.

getLikedTweets

  • Fetches liked tweets from a Twitter user. Requires authentication.

    Parameters

    • user: string

      The user whose likes should be returned.

    • maxTweets: number = 200

      The maximum number of tweets to return. Defaults to 200.

    Returns AsyncGenerator<Tweet, any, unknown>

    An AsyncGenerator of liked tweets from the provided user.

getProfile

  • Fetches a Twitter profile.

    Parameters

    • username: string

      The Twitter username of the profile to fetch, without an @ at the beginning.

    Returns Promise<Profile>

    The requested Profile.

getTrends

  • Fetches the current trends from Twitter.

    Returns Promise<string[]>

    The current list of trends.

getTweet

  • Fetches a single tweet.

    Parameters

    • id: string

      The ID of the tweet to fetch.

    Returns Promise<null | Tweet>

    The Tweet object, or null if it couldn't be fetched.

getTweetWhere

  • Fetches the first tweet matching the given query.

    Example:

    const timeline = scraper.getTweets('user', 200);
    const retweet = await scraper.getTweetWhere(timeline, { isRetweet: true });

    Parameters

    • tweets: AsyncIterable<Tweet>

      The AsyncIterable of tweets to search through.

    • query: TweetQuery

      A query to test all tweets against. This may be either an object of key/value pairs or a predicate. If this query is an object, all key/value pairs must match a Tweet for it to be returned. If this query is a predicate, it must resolve to true for a Tweet to be returned.

      • All keys are optional.
      • If specified, the key must be implemented by that of Tweet.

    Returns Promise<null | Tweet>

getTweets

  • Fetches tweets from a Twitter user.

    Parameters

    • user: string

      The user whose tweets should be returned.

    • maxTweets: number = 200

      The maximum number of tweets to return. Defaults to 200.

    Returns AsyncGenerator<Tweet, any, unknown>

    An AsyncGenerator of tweets from the provided user.

getTweetsAndReplies

  • Fetches tweets and replies from a Twitter user.

    Parameters

    • user: string

      The user whose tweets should be returned.

    • maxTweets: number = 200

      The maximum number of tweets to return. Defaults to 200.

    Returns AsyncGenerator<Tweet, any, unknown>

    An AsyncGenerator of tweets from the provided user.

getTweetsAndRepliesByUserId

  • Fetches tweets and replies from a Twitter user using their ID.

    Parameters

    • userId: string

      The user whose tweets should be returned.

    • maxTweets: number = 200

      The maximum number of tweets to return. Defaults to 200.

    Returns AsyncGenerator<Tweet, void, unknown>

    An AsyncGenerator of tweets from the provided user.

getTweetsByUserId

  • Fetches tweets from a Twitter user using their ID.

    Parameters

    • userId: string

      The user whose tweets should be returned.

    • maxTweets: number = 200

      The maximum number of tweets to return. Defaults to 200.

    Returns AsyncGenerator<Tweet, void, unknown>

    An AsyncGenerator of tweets from the provided user.

getTweetsWhere

  • Fetches all tweets matching the given query.

    Example:

    const timeline = scraper.getTweets('user', 200);
    const retweets = await scraper.getTweetsWhere(timeline, { isRetweet: true });

    Parameters

    • tweets: AsyncIterable<Tweet>

      The AsyncIterable of tweets to search through.

    • query: TweetQuery

      A query to test all tweets against. This may be either an object of key/value pairs or a predicate. If this query is an object, all key/value pairs must match a Tweet for it to be returned. If this query is a predicate, it must resolve to true for a Tweet to be returned.

      • All keys are optional.
      • If specified, the key must be implemented by that of Tweet.

    Returns Promise<Tweet[]>

getUserIdByScreenName

  • Fetches the user ID corresponding to the provided screen name.

    Parameters

    • screenName: string

      The Twitter screen name of the profile to fetch.

    Returns Promise<string>

    The ID of the corresponding account.

Private handleResponse

hasGuestToken

  • Returns if the scraper has a guest token. The token may not be valid.

    Returns boolean

    true if the scraper has a guest token; otherwise false.

isLoggedIn

  • Returns if the scraper is logged in as a real user.

    Returns Promise<boolean>

    true if the scraper is logged in with a real user account; otherwise false.

login

  • Login to Twitter as a real Twitter account. This enables running searches.

    Parameters

    • username: string

      The username of the Twitter account to login with.

    • password: string

      The password of the Twitter account to login with.

    • Optional email: string

      The email to log in with, if you have email confirmation enabled.

    • Optional twoFactorSecret: string

      The secret to generate two factor authentication tokens with, if you have two factor authentication enabled.

    Returns Promise<void>

logout

  • Log out of Twitter.

    Returns Promise<void>

searchProfiles

  • Fetches profiles from Twitter.

    Parameters

    • query: string

      The search query. Any Twitter-compatible query format can be used.

    • maxProfiles: number

      The maximum number of profiles to return.

    Returns AsyncGenerator<Profile, void, unknown>

    An AsyncGenerator of tweets matching the provided filter(s).

searchTweets

  • Fetches tweets from Twitter.

    Parameters

    • query: string

      The search query. Any Twitter-compatible query format can be used.

    • maxTweets: number

      The maximum number of tweets to return.

    • searchMode: SearchMode = SearchMode.Top

      The category filter to apply to the search. Defaults to Top.

    Returns AsyncGenerator<Tweet, void, unknown>

    An AsyncGenerator of tweets matching the provided filters.

setCookies

  • Set cookies for the current session.

    Parameters

    • cookies: (string | Cookie)[]

      The cookies to set for the current session.

    Returns Promise<void>

withCookie

  • Sets the optional cookie to be used in requests.

    Parameters

    • _cookie: string

      The cookie to be used in requests.

    Returns Scraper

    This scraper instance.

    Deprecated

    This function no longer represents any part of Twitter's auth flow.

withXCsrfToken

  • Sets the optional CSRF token to be used in requests.

    Parameters

    • _token: string

      The CSRF token to be used in requests.

    Returns Scraper

    This scraper instance.

    Deprecated

    This function no longer represents any part of Twitter's auth flow.

Settings

Member Visibility

Theme

On This Page