Our n8n workflow monitors 42 airline X accounts every 30 minutes and feeds disruption alerts into a WordPress liveblog covering MENA travel. Last month, the Apify actor powering it silently started returning zero results. This is what happened next — including one actor that turned out to be a scam, a 13× pricing trap hiding in the author’s own upgrade suggestion, and a parameter behaviour that only revealed itself through controlled testing.
The whole rebuild took one Claude Code session with the Apify agent skill installed. Here’s the full breakdown.
What Broke and Why
The original setup used apidojo/tweet-scraper with 29 separate n8n HTTP Request nodes — one per airline account. Each node called the Apify API independently, fetching the latest 50 tweets from a single account. It worked, until it didn’t.
Apify logs showed a rate limit message citing a “terms of service violation.” The root cause: our usage pattern — many short-interval queries each returning very few tweets — matched the actor’s restricted usage patterns. The actor is optimised for bulk scraping (~50+ tweets per query) and flags low-yield monitoring as misuse. The actor author’s suggested fix: switch to apidojo/twitter-scraper-lite, branded as “Twitter Scraper Unlimited: No Limits.”
There was also a structural flaw in the original design. Because we used a shared 50-tweet cap across all accounts in a single task, high-frequency accounts like @flightradar24 would consume the cap before low-frequency airline accounts appeared. Some airlines were consistently missed on every run — we just hadn’t noticed until the actor broke entirely and we started looking closely.
The Pricing Trap in the Suggested Upgrade
Before testing the suggested replacement, Claude Code fetched its full pricing structure via the Apify API. What came back was a problem.
apidojo/twitter-scraper-lite uses per-query pricing: $0.016 per profile query, plus per-item tiers above the first ~40 results. For bulk scraping a small number of high-volume accounts, that’s reasonable. For our pattern — 42 accounts, each returning 2–3 tweets per 30-minute monitoring run — it’s catastrophic:
| Metric | Author’s suggestion (apidojo-lite) | Our chosen actor (gentle_cloud) |
|---|---|---|
| Pricing model | $0.016 per profile query + per-item tiers | Flat $0.05 per run (all accounts) |
| Cost per run (42 accounts) | 42 × $0.016 = $0.672 | $0.05 |
| Daily cost (48 runs) | $32.26 | $2.40 |
| Projected monthly | ~$968 | ~$72 |
The “Unlimited” actor is built for queries against a small number of high-volume accounts. Our use case is the inverse: many accounts, few tweets each. We’d pay the full $0.016 query fee per account and retrieve 2–3 tweets — 13× the cost of the old setup for identical data. The word “Unlimited” refers to rate limits, not to how well its pricing model fits a monitoring pattern.
How Claude Code with the Apify Skill Changed the Approach
Instead of manually browsing the Apify marketplace, reading actor documentation, configuring test tasks in the console, and running tests via curl, I installed the Apify agent skill:
Terminalnpx skills add apify/agent-skills
The skill gives Claude Code direct API access to the Apify platform: searching actors with filters (category, pricing model, success rate, monthly users), creating and configuring tasks with specific input parameters, running actors and retrieving results in real time, inspecting dataset schemas, and pulling run history with cost breakdowns. The entire research-test-iterate cycle happened inside the conversation.
Without the skill, the process would have been: browse marketplace → read docs → configure test in Apify console or Postman → run, wait, download dataset → inspect manually → repeat for each candidate. That’s a multi-day cycle across multiple tools. With the skill, Claude ran all of it programmatically and I steered the decisions. The pricing analysis of the author’s suggested actor — fetching nested JSON pricing tiers from the API and calculating the cost for our specific query volume — is a good example of something that would have taken significant manual effort and probably a spreadsheet.
Attempt 1: The Actor That Injected Fake Data
Claude’s first recommendation was kaitoeasyapi/twitter-x-data-tweet-scraper-pay-per-result-cheapest — 1,404 monthly users, 99.7% success rate, pay-per-result pricing at $0.0005/tweet. The metrics looked clean.
Before testing, I asked two questions. How does the from: search term work — is it pulling from official accounts only? Claude confirmed it uses Twitter’s native advanced search syntax, so from:qatarairways returns only tweets by that account. And what’s the operational risk of lower-usage actors? The answer: reliability and schema consistency, not security — Apify actors run in sandboxed containers regardless of the author.
Claude configured a task and ran it. I connected it to n8n and pushed results through my normalize node. Output: zero real tweets. Every item was an advertising message from the actor author.
What the Apify logs revealed: the actor’s query builder had failed silently —
"query is :"(empty string). It returned 0 real results, then injected 15 fake items containing the actor author’s billing policy to charge a minimum fee. You get charged even when the actor returns no useful data, because the fake advertising items count as “results.”
99.7% success rate. Zero actual data. Marketplace metrics don’t capture billing behaviour — that only shows up when you run controlled tests and inspect the dataset.
Attempt 2: The Actor That Works
Claude immediately pivoted to gentle_cloud/twitter-tweets-scraper. URL-based input instead of query syntax — you pass profile URLs directly. 99.2% success rate. Categorised as “News” in the Apify marketplace, which is precisely the monitoring use case. Transparent flat-rate billing at $0.05 per run regardless of account count. No minimum charge padding.
Claude configured a task and ran test queries. Output was clean: real tweets, consistent schema, no fake data. This was the one.
The Hidden Behaviour: result_count Is a Total Cap
Claude’s initial configuration assumed result_count was per-account. It isn’t. After running several test configurations from n8n and sharing the normalize output, we analysed five runs via the Apify API and confirmed it’s a total cap across all accounts in the batch.
| result_count | Accounts | Items returned | What happened |
|---|---|---|---|
"50" |
29 | 50 | Cap hit — only 12 accounts represented |
"10" |
29 | 10 | @flightradar24 took 6 of 10 slots |
"30" |
3 | 14 | Cap not hit — all available tweets returned |
With a low cap, high-frequency accounts dominate — the exact same coverage failure we’d had in the old setup. My first instinct was to split into two Build Input nodes: one for aggregator accounts with a tight limit, another for airlines with a higher limit. Then I stepped back. Flightradar24 and aviationbrk are third-party aggregators, not official airline communications channels. Removing them entirely eliminated the cap domination problem and kept the feed focused on what it’s actually for: official airline disruption communications.
With result_count set to 100 and 42 pure-airline accounts, every active account gets representation in every run. The documentation doesn’t say whether the cap is per-account or global — the only way to find out was running controlled tests.
Final Configuration: 2 Nodes, 42 Airlines
The workflow went from 29 HTTP Request nodes to 2. Node 1 is a Code node that builds the input; Node 2 is a single HTTP Request to the Apify run endpoint.
n8n · Node 1 — Build Input (Code node)const handles = [
// Middle East (15)
'qatarairways','emirates','etihad','flydubai','airarabia',
'omanair','GulfAir','SalamAir','kuwaitairways','Saudi_Airlines',
'flynas','RoyalJordanian','JazeeraAirways','flyadeal','RiyadhAir',
// South Asia (6)
'IndiGo6E','AirIndiaExpress','SpiceJet','airindia','Official_PIA','BimanbdAirline',
// Europe (7)
'British_Airways','TurkishAirlines','airfranceen','lufthansa',
'wizzair','PegasusAirlines','SAS',
// Africa (2)
'EgyptAir','flyethiopian',
// Asia-Pacific (8)
'SingaporeAir','cathaypacific','koreanair','MalaysiaAirlines',
'CebuPacificAir','MyanmarNationalAirlines','flyPAL','XiamenAir',
// Americas & China (4)
'AirCanada','AmericanAir','airchina','ChinaEastern'
];
const yesterday = new Date();
yesterday.setDate(yesterday.getDate() - 1);
return [{ json: {
start_urls: handles.map(h => ({ url: `https://x.com/${h}` })),
since_date: yesterday.toISOString().split('T')[0],
result_count: '100'
}}];
n8n · Node 2 — Fetch Tweets (HTTP Request node)Method: POST
URL: https://api.apify.com/v2/acts/gentle_cloud~twitter-tweets-scraper/run-sync-get-dataset-items
Body: JSON from Node 1
Timeout: 310s (actor-side timeout: 300s)
Retry: 2 attempts, 5s wait
One Apify call. One response. Passes directly into the existing Normalize Tweets node with no schema changes required.
The Full Cost Comparison
| Old (apidojo) | Suggested (apidojo-lite) | Scam (kaitoeasyapi) | New (gentle_cloud) | |
|---|---|---|---|---|
| Pricing model | Per dataset item | Per query + per item | Per result + min charge | Flat per run |
| Cost per execution | $0.004 × 29 = $0.12 | $0.016 × 42 = $0.67 | Charged for fake data | $0.05 |
| Daily (48 runs) | ~$2.42 | ~$32.26 | N/A | ~$2.40 |
| Monthly | ~$72 | ~$968 | More + garbage data | ~$72 |
| Accounts covered | 29 | 42 (if affordable) | 0 real data | 42 |
| Status | Broken (ToS flag) | 13× too expensive | Fake data injection | 100% success rate |
Same daily spend as the old setup ($2.40 vs $2.42). 45% more airline coverage (42 vs 29). Two nodes instead of twenty-nine. Reply filtering built in — verified at zero replies across 89 tweets in five test runs.
Five Things Worth Knowing Before You Ship an Apify Integration
- Test billing behaviour before committing to production. kaitoeasyapi had strong marketplace metrics — 1,400+ monthly users, 99.7% success rate. It still injected fake data to charge minimum fees on empty results. Metrics reflect usage volume and run success, not billing honesty.
- Don’t follow upgrade suggestions without modelling the cost for your actual query pattern. The author’s suggested replacement was genuinely redesigned for monitoring — its changelog says so explicitly. But its per-query pricing made it 13× more expensive for a many-accounts-few-tweets pattern. The advertised use case and your actual use case can have very different economics.
- Verify whether result caps are per-account or global through controlled testing. Documentation may not say. Run the same actor with varying account counts and cap values, compare actual item counts, and you’ll know. There’s no shortcut.
- Simplify before you optimise. When @flightradar24 was dominating the result cap, the instinct was to build per-account limiting logic. The right answer was removing aggregators from an official-airline-only feed. Fewer moving parts means fewer failure modes — and a more coherent data source.
- Agent skills collapse multi-day research cycles into single sessions. The Apify skill let Claude search the marketplace, create tasks, run actors, inspect datasets, and pull cost data without leaving the conversation. The pricing analysis alone — fetching nested JSON pricing tiers and modelling costs against our specific run configuration — would have taken an hour manually. It took one API call.
The n8n workflow JSON for the 2-node setup is saved as twitter-monitor-2-nodes.json. The actor is gentle_cloud/twitter-tweets-scraper. The Apify skill installs with npx skills add apify/agent-skills. If you’re building something similar and want to talk through the architecture, reach out.