How to rehydrate tweet IDs from X in bulk

#What it means to rehydrate tweet IDs

Rehydrating tweet IDs is looking each ID up again to get the full tweet behind it: the text, the posting time, the likes and views, and the author's profile. A tweet ID is the 19-digit number at the end of every post link, and a list of them is called a dehydrated dataset. To see when any single ID was posted before you spend a request on it, paste it into the free Tweet ID to Date Converter.

Datasets travel as bare IDs because of X's rules. The X Developer Policy (opens in a new tab) lets you share post IDs and user IDs with others, but not the tweets themselves, and caps what you hand any one recipient at 1.5M post IDs every 30 days without X's written permission. So almost every list you will meet is numbers only:

• Research archives. Election, crisis and COVID-19 collections on Harvard Dataverse, Zenodo or GitHub ship as text files of IDs, often tens of millions of lines.
• Social-listening exports. Meltwater's API, for one, exports X posts as an ID and an author ID only and tells customers to rehydrate them (opens in a new tab).
• Your own records. Old databases, logs and spreadsheets that kept the ID and dropped the text.

Rehydration returns the tweet as it is today. A post deleted since collection, or one from an account that went protected or was suspended, comes back empty, and the metrics and author profile you get are current values, not the ones from the day it was posted. That is why every rehydrated set is a little smaller than the list it started from.

#What it costs and how long it takes

Rehydrating a million tweet IDs costs $5,000 on the official X API and from $14 on Sorsa API, and both look up 100 IDs per call. The difference is the unit: X's rate card (opens in a new tab) bills every post it returns, $0.005 each, while Sorsa API bills one request per batch of 100, whatever is inside.

Two more rules shape the official X API's price for big lists. Pay-per-use stops at 3M post reads per monthly billing cycle; a larger list either waits for next month or needs an Enterprise contract, from about $42,000 a month. And from November 22, 2026, X bills every object that rides along through expansions (opens in a new tab), so each author profile you ask for adds $0.010 to the post's $0.005.

• 10K

  Sorsa API

  free (the 100 signup requests)

  Sorsa API time at 20 requests a second

  about 5 seconds

  Official X API

  $50

• 100K

  Sorsa API

  from $1.40

  Sorsa API time at 20 requests a second

  under a minute

  Official X API

  $500

• 1M

  Sorsa API

  from $14

  Sorsa API time at 20 requests a second

  about 8 minutes

  Official X API

  $5,000

• 3M

  Sorsa API

  from $42

  Sorsa API time at 20 requests a second

  about 25 minutes

  Official X API

  $15,000, the whole monthly cap

• 10M

  Sorsa API

  from $140

  Sorsa API time at 20 requests a second

  about 1 hour 23 minutes

  Official X API

  Enterprise only, $42K+ a month

Sorsa API's "from" prices use its lowest rate per request; at the Pro plan's rate a million IDs cost $19.90. Failed requests are never charged, and the author's full profile comes inside every tweet at no extra cost. For the same comparison on fresh collection by search, see what 1 million tweets cost.

#Rehydrate tweet IDs with Sorsa API

To rehydrate tweet IDs at scale, use Sorsa API, a Twitter (X) API and scraper: POST /tweet-info-bulk (opens in a new tab) takes up to 100 IDs or post links per request and returns each tweet with its author's full profile, from $0.014 per 1,000 IDs at 20 requests per second on every plan. It needs one API key, no X developer account, no OAuth, and never your X login.

1. Get a key. Sign up and copy the API key from the dashboard. The 100 free requests rehydrate up to 10K IDs.

2. Send a batch. Put up to 100 IDs, as strings, in the tweet_links array. Full post links work too.

   curl -X POST "https://api.sorsa.io/v3/tweet-info-bulk" \
     -H "ApiKey: YOUR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"tweet_links": ["1856270992012337728",
          "1399991672632369728", "1496699442164785728"]}'
   

3. Read the tweets. The response is one tweets array, not paginated. Each tweet carries its own id, which is how you match it to the ID you sent.

   {
     "tweets": [
       {
         "id": "1856270992012337728",
         "full_text": "Day 3 of the field survey: river levels are normal.",
         "created_at": "2024-11-12T09:41:07Z",
         "lang": "en",
         "likes_count": 42,
         "retweet_count": 7,
         "reply_count": 3,
         "quote_count": 1,
         "view_count": 5210,
         "bookmark_count": 2,
         "conversation_id_str": "1856270992012337728",
         "is_reply": false,
         "is_quote_status": false,
         "user": {
           "id": "1497281610093821952",
           "username": "field_notes_demo01",
           "display_name": "Field Notes Demo",
           "followers_count": 1288,
           "protected": false,
           "verified": false
         }
       }
     ]
   }
   

   Show full response

A 403 means the key is missing or its requests have run out; a 429 means more than 20 requests in one second, so wait a second and send the batch again. Quotes and reposts carry the original tweet nested inside (quoted_status, retweeted_status), a reply names the tweet it answers in in_reply_to_tweet_id, and media links sit in entities.

Good to know

Rehydrated metrics are today's numbers. A tweet from 2016 comes back with the likes and views it has now and with its author's current name, bio and follower count. If your study needs the values from the day of posting, use the counts stored in the original dataset, when it has them.

#Clean the ID list before you spend a request

Most failed rehydrations start in the ID file, not in the API. A tweet ID has 19 digits, more than a spreadsheet or a 64-bit float keeps, so a list that passed through Excel, pandas with a float column or JavaScript's JSON.parse points to tweets that don't exist. A well-known Stack Overflow question about getting back 18 tweets out of 24K came down to a damaged ID list, not to the API.

Three rules keep the list whole:

• Read IDs as text. In pandas, pd.read_csv(path, dtype=str); in a JSON file, keep the id_str or string form; never open the raw file in Excel.
• Drop duplicates first. Shared datasets often repeat IDs across daily files, and every duplicate wastes a slot in a batch of 100.
• Spot rounded lines. An ID in scientific notation (1.85627E+18) or ending in a run of zeros was cut by a spreadsheet. Get those lines again from the original source; no API can repair them.

The planner below runs those checks on a sample of your list in the browser and prices the full job.

#A resumable job for millions of tweet IDs

A list of millions of IDs needs a job that runs at the full rate and survives a crash without paying twice. The script below reads the IDs as text, drops duplicates, sends 20 batches of 100 every second, writes tweets as they arrive, and records each finished batch in progress.txt, so a rerun starts where the last one stopped.

import json
import os
import time
from concurrent.futures import ThreadPoolExecutor

import requests

URL = "https://api.sorsa.io/v3/tweet-info-bulk"
HEADERS = {"ApiKey": os.environ.get("SORSA_API_KEY", "YOUR_API_KEY")}
BATCH, PER_SECOND = 100, 20


def load_ids(path):
    seen, ids = set(), []
    with open(path, encoding="utf-8") as f:
        for line in f:
            tid = line.strip().split("/status/")[-1].split("?")[0]
            if tid.isdigit() and tid not in seen:
                seen.add(tid)
                ids.append(tid)
    return ids


def lookup(batch):
    for attempt in range(5):
        r = requests.post(URL, headers=HEADERS, timeout=60,
                          json={"tweet_links": batch})
        if r.status_code == 429 or r.status_code >= 500:
            time.sleep(2 ** attempt)
            continue
        if r.status_code == 404:
            return []
        r.raise_for_status()
        return r.json().get("tweets", [])
    raise RuntimeError(f"batch starting {batch[0]} failed 5 times")


def run(ids):
    batches = [ids[i:i + BATCH] for i in range(0, len(ids), BATCH)]
    start = 0
    if os.path.exists("progress.txt"):
        start = int(open("progress.txt").read() or 0)
    with open("tweets.jsonl", "a", encoding="utf-8") as out, \
         open("missing_ids.txt", "a", encoding="utf-8") as miss, \
         ThreadPoolExecutor(max_workers=PER_SECOND) as pool:
        for k in range(start, len(batches), PER_SECOND):
            t0 = time.monotonic()
            group = batches[k:k + PER_SECOND]
            for batch, tweets in zip(group, pool.map(lookup, group)):
                got = {t["id"] for t in tweets}
                for t in tweets:
                    out.write(json.dumps(t, ensure_ascii=False) + "\n")
                for tid in batch:
                    if tid not in got:
                        miss.write(tid + "\n")
            out.flush()
            miss.flush()
            with open("progress.txt", "w") as p:
                p.write(str(k + len(group)))
            time.sleep(max(0, 1 - (time.monotonic() - t0)))


if __name__ == "__main__":
    ids = load_ids("tweet_ids.txt")
    run(ids)
    with open("tweets.jsonl", encoding="utf-8") as f:
        got = {json.loads(line)["id"] for line in f}
    print(f"recovered {len(got):,} of {len(ids):,} IDs "
          f"({len(got) / len(ids):.1%})")

Set the key with export SORSA_API_KEY=..., put one ID or post link per line in tweet_ids.txt, and run python rehydrate.py. At 20 requests a second, a million IDs finish in about 8 minutes and ten million in under an hour and a half. Before a long run, GET /key-usage-info (opens in a new tab) shows how many requests the key has left, and that check is free.

#Why some tweets don't come back

A tweet ID returns nothing when the tweet no longer exists for the public: its author deleted it, the account turned protected, or the account was suspended or deactivated. No API brings those back; an ID that returns nothing is an answer, not an error, so the script writes it to missing_ids.txt and moves on.

How much goes missing depends on the age and topic of the list. A set collected last month may come back almost whole, while years-old sets on elections, conflicts or spam-heavy hashtags lose far more, because those accounts are suspended or delete more often. Measure it instead of guessing:

• Recovery rate. Tweets returned divided by unique IDs sent, the last line the script prints. Report it with the date you rehydrated, since the same list returns fewer tweets every year.
• Missing by period. Decode the dates of the missing IDs (the planner above and the ID-to-date tool both read them) to see whether losses cluster in one week or one event.
• Damaged versus gone. Rounded IDs show up as missing too. Check the missing file for runs of zeros before you count them as deleted.

For a published study, keep the ID list you started from, the recovery rate and the rehydration date together: the next team to rehydrate the same list will get a different set, and those three facts explain the difference.

#Hydrator, twarc and other tools in 2026

The tools that used to rehydrate datasets all ran on X's own API access, and most of that access is gone. DocNow's Hydrator, the desktop app many research guides still point to, now opens its GitHub page with a notice that the application keys it relied on were rescinded and the app is no longer useful.

twarc's hydrate command still works, but only with your own bearer token from the official X API, so it bills at X's pay-per-use rate: $5,000 per million posts, inside the 3M monthly cap. R packages such as rehydratoR and academictwitteR were written for free access levels and an Academic Research track that X closed in 2023. To try a few batches by hand first, the API playground runs the same endpoint with your key and exports CSV. If you have a twarc pipeline already, the JSON you need downstream is close to what POST /tweet-info-bulk returns, so the swap is the request function, not the whole pipeline.

For research teams, Sorsa API also offers academic access, tailored to each project and requested from an institutional email. Rehydration is one step of a larger build: collecting by date range is covered in historical Twitter data, and turning a rehydrated set into labelled training data in how to build a Twitter dataset for machine learning.

#Rehydrate your first 10K tweet IDs

Rehydrating a list of tweet IDs is a batch job of 100 IDs per call, and the only real choices are the price and the speed. On Sorsa API a million IDs cost from $14 and take about 8 minutes, against $5,000 and a 3M monthly cap on the official X API. Clean a sample of your list in the planner, run the first 10K IDs on the 100 free requests, and check the recovery rate before you send the rest.

#FAQ