Nothing new. Nitter for example has been using these endpoints for a while.
Maybe there's just not enough engineering talent left at Twitter to do it.
31–40 of 41 posts
Nothing new. Nitter for example has been using these endpoints for a while.
Maybe there's just not enough engineering talent left at Twitter to do it.
If you want to use it then you should seriously limit the usage. The rate limits are there to protect the servers but there are things that will notice that you are overall using the system at a rate that an actual person would not use and will ban you.
If this can be built, build a twitter clone. As each big entity/celebrity quits twitter or starts having serious conflicts, approach them to cross-post their content to the new clone site. In a year the masses will follow. Once Musk misses a billion-dollar-per-year payments a few times to the Saudis they will own twitter and then it will be like TikTok censorship the next time they murder a journalist they disagree w…
If this can be built, build a twitter clone. As each big entity/celebrity quits twitter or starts having serious conflicts, approach them to cross-post their content to the new clone site. In a year the masses will follow. Once Musk misses a billion-dollar-per-year payments a few times to the Saudis they will own twitter and then it will be like TikTok censorship the next time they murder a journalist they disagree w…
Do you really expect the richest man in the world to miss payments? He has enough money to buy Twitter 4 more times
This was inevitably going to get more popular after the new API pricing was announced. Unfortunately this also means they're going to start being more aggressive on bot detection. We may soon find ourselves needing to run headless browsers to scrape data, like with TikTok: https://nullpt.rs/reverse-engineering-tiktok-vm-1
These puppeteer, playwright wrappers make bots so easy. And you can distribute so easy too. Back in my day (IE6) it wasn't quite so simple. Trying to make your site undcrape-able while still readable is (still) impossible.
Earlier quoted context omitted.
How is it "stealing" to access a service you already have access to for free? Go to twitter.com in a browser. This is just a different type of browser. Physical analogies never work right with digital data.
I'm sure companies wouldn't mind if a singular person was doing singular things with the API, but when someone uses this API to behave like more than 1 person is where companies get annoyed. To use an annoying analogy or two, take a penny leave a penny but someone shoves their hand in there and takes all of it. It costs money to service requests and twitter et all make that money back through advertising and subscrip…
Earlier quoted context omitted.
It continues to surprise me that Elon hasn't shut out Nitter yet. Maybe there's just not enough engineering talent left at Twitter to do it.
There’s more engineering talent now not less
Earlier quoted context omitted.
For some reason Twitter’s frontend uses a hard-coded bearer token, at least for anonymous users. You’ll see exactly the same string if you load a Twitter page and look at the XHR requests in your own browser. (It seems to change occasionally, but old ones keep working in my experience.)
FWIW, I have never logged in to Twitter and I have always been able to retrieve all tweets. At first, I used mobile.twitter.com in a text-only browser, no token required. Since they started using GraphQL, I retrieve tweets as JSON. They have changed the token once. The current one is Bearer AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA IME, the old token will…
#!/bin/sh
test $# -gt 0||exec echo usage: $0 query
DATA=$(echo '{"query":"'$@'","analyticsTags":["web"],"page":0,"hitsPerPage":30,"minWordSizefor1Typo":4,"minWordSizefor2Typos":8,"advancedSyntax":true,"ignorePlurals":false,"clickAnalytics":true,"minProximity":7,"numericFilters":[],"tagFilters":["story",[]],"typoTolerance":"min","queryType":"prefixNone","restrictSearchableAttributes":["title","comment_text","url","story_text","author"],"getRankingInfo":true}');
HOST=uj5wyc0l7x-3.algolianet.com
_PATH="/1/indexes/Item_production_sort_date/query?x-algolia-agent=Algolia%20for%20JavaScript%20(4.0.2)%3B%20Browser%20(lite)&x-algolia-api-key=8ece23f8eb07cd25d40262a1764599b1&x-algolia-application-id=UJ5WYC0L7X"
# HTTP client (curl)
#curl -A "" -d "$DATA" "https://$HOST$_PATH"
# TCP client
#echo "
#foreground=no
#[x]
#accept=127.0.0.8:80
#client=yes
#connect=167.114.119.142:443
#options=NO_TICKET
#options=NO_RENEGOTIATION
#renegotiation=no
#sni=
#sslVersion=TLSv1.3
#" |stunnel -fd 0;
#tr @ '\r' Earlier quoted context omitted.
For some reason Twitter’s frontend uses a hard-coded bearer token, at least for anonymous users. You’ll see exactly the same string if you load a Twitter page and look at the XHR requests in your own browser. (It seems to change occasionally, but old ones keep working in my experience.)
FWIW, I have never logged in to Twitter and I have always been able to retrieve all tweets. At first, I used mobile.twitter.com in a text-only browser, no token required. Since they started using GraphQL, I retrieve tweets as JSON. They have changed the token once. The current one is Bearer AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA IME, the old token will…
Everyone, including any member of the public, who visits twitter.com gets the same Bearer token.
No need to have an "account" with Twitter or to be "logged in".
One can simulate this with cURL.
js=$(curl -sA "" https://twitter.com|grep -m1 -o "https://abs.twimg.com/responsive-web/client-web-legacy/main[^\"]*");
curl -A "" $js|tr , '\n'|grep -o \"AAAA.*\"
The same Bearer token value is used by people around the web for retrieving public tweets. It's public information. For example,https://stackoverflow.com/questions/61140863/python-download...
https://github.com/twintproject/twint/raw/master/twint/run.p...
https://pypi.org/project/ScrapeTweets/
https://stackoverflow.com/questions/67137294/twitter-scrapin...
https://github.com/m4fn3/pytweetdeck/blob/master/pytweetdeck...
https://github.com/jonbakerfish/TweetScraper/issues/127
https://github.com/JustAnotherArchivist/snscrape/issues/536
https://gist.github.com/codemasher/67ba24cee88029a3278c87ff9...
https://github.com/HoloArchivists/twspace-dl/issues/26
https://gist.github.com/AzureFlow/01cff883b9f1b22e8d0c094df9...
https://greasyfork.org/hu/scripts/454409-video-downloader-fo...
https://gist.github.com/moxak/ed83dd4169112a0b1669500fe85510...
https://gist.github.com/ceres-c/7c16a40c10cb476cce2c4b902334...
https://gist.github.com/theowenyoung/d4a62746025f7af8cdd8bfb...