Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dorking: the use of search engines to find very specific data

alec.fyi

Post reply on HN
Print view View original

121–130 of 207 posts

Re: Dorking: the use of search engines to find very specific data

#121

Post by jlokier »

solarist wrote: post #99
GordonS wrote: post #75

Earlier quoted context omitted.

Worth mentioning that even if you put a term in double quotes, Google still tries to be too clever - you are not guaranteed to get results that contain your quotes search term :/

As a workaround and under search tools one can enable the “verbatim” option.

I was under the impression "verbatim" is to disable filter-bubble personalisation.

Normal queries are tailored to your personal filter bubble. You can't see what other people see from same search, and if you're doing SEO or just trying to find who tends to come top in results for something you have a lot of history looking at, you can't tell who comes top for other people.

Re: Dorking: the use of search engines to find very specific data

#122

Post by smichel17 »

flywheel wrote: post #77
efreak wrote: post #67

Earlier quoted context omitted.

In this case, depreciated is incorrect. Removal has already happened, the "period of time" is already over.

Removing APIs is not a great practice though. Look at AWS, they version their APIs, they don't just remove them, and removing them should be unnecessary if your underlying tech isn't brittle and badly written. "Depreciated" is a far better term to use, with a far better outcome in my opinion. Companies that remove old versions of APIs and break existing client apps (that possibly can't be udpated) really suck.

I don't think these two are incompatible?

If APIv3 has a `/foo` endpoint that is deprecated, usually I take that to mean that the developers discourage its use, and likely plan to remove it in a future version (say, APIv4 or APIv5). `/foo` will never be removed from APIv3, because that would be a breaking change, and so if I'm willing to stay on v3 forever, that's fine, but in the (likely) event I will want to take advantage of new features at some point in the future, I'm doing myself a disservice by using /foo because it will make the migration harder.

There is at least one case where I think "deprecated" is clearly, inarguably, the right word: when the developer wants to remove a part of an API (say, because it is a large maintenance burden), but it's also committed to stability, so they won't remove that api until some acceptably small number of users are using it.

Re: Dorking: the use of search engines to find very specific data

#123

Post by bloaf »

IggleSniggle wrote: post #104
userbinator wrote: post #95

Earlier quoted context omitted.

That really angers me, and I've tripped it more times than I can count, usually by searching for very specific things. Coworkers have also run into it multiple times (before everyone started working from home, we would exclaim "Fuck you, Google!" and raise a middle finger to the screen, which was a cue to everyone else to help). The fact that they think you're "not human" when you use a search engine for its intended…

I’m curious about this. Can you give an example of the kind of query you are talking about where Google assumes you are a bot and not a human?

I get them all the dang time

Suppose I want to search for a textbook in an open directory. This query will get flagged almost immediately:

    textbook -inurl:htm -inurl:html -inurl:asp intitle:"index of" +(epub|mobi|pdf|txt)
Suppose someone says to me "no one in this forum believes (dumb conspiracy theory from Obama-times)" or "the lamestream media never reported on X." Naturally I will do a few quick iterations of

    conspiracy related "terms" site:someforum.com before:2017-01-01
which will trigger a humanity check if you iterate too quickly.

I got hit recently when the news about Tucker Carlson's racist writer came out. I set out to find the full threads in which the offending comments were made. Iterating through combinations of text, usernames, and urls like

    "full text of offensive post" username before:date-news-broke -inurl:news
got me checked every 3rd search or so.

Re: Dorking: the use of search engines to find very specific data

#126

Post by jon_richards »

GordonS wrote: post #101
devjungle wrote: post #91

Earlier quoted context omitted.

This must be a recent change? It's been driving me nuts lately. I have to resort to adding a lot of negated search terms to compensate but it's still sub optimal.

No, this has been the case for a long time, years anyway. I don't know if it goes back quite as far as when they removed the '+' operator tho. But bejesus, this drives me nuts! If I know the double quotes function even exists, then Google should know I actually want to use it as intended - it shouldn't decide "yeah, but maybe you'd like these irrelevant results too!"

I think it’s primarily for people copying and pasting something like an error message, which may have user-specific data in quotes. That should be what “not many results for...” is for, but Google is always trying to optimize away those clicks. Maybe they could add a “programatic” search feature with documented syntax for power users.

Re: Dorking: the use of search engines to find very specific data

#127

Post by userbinator »

uj8efdkjfdshf wrote: post #120
userbinator wrote: post #95

Earlier quoted context omitted.

That really angers me, and I've tripped it more times than I can count, usually by searching for very specific things. Coworkers have also run into it multiple times (before everyone started working from home, we would exclaim "Fuck you, Google!" and raise a middle finger to the screen, which was a cue to everyone else to help). The fact that they think you're "not human" when you use a search engine for its intended…

You could always keep Google Chrome open to rerun these specific queries, as the captchas are less irritating to solve then.

I would sooner switch to DDG, Bing, or Yahoo, or even just change my IP, than use Chrome.

The ban seems to expire after a short time too.

Re: Dorking: the use of search engines to find very specific data

#128

Post by 1vuio0pswjnm7 »

abarrettwilsdon wrote: post #119
1vuio0pswjnm7 wrote: post #113

I have a question for anyone reading this thread: Do you believe you can get consistent results with any search? For example, if we pick some uncommon search terms will we get the same results on the first search, the second search, the third, etc. Or will the results change? I did a search with some terms from one of the comments in this thread, in quotes. The first search returned only one result: this thread. As I…

If it's specific enough, the SERP should stay the same until someone else publishes the same thing e.g. the search of another article "set up Google Sheets APIs (and treat Sheets like a database)" turns up my site and a couple Twitter threads talking about it (plus a phishing site which has scraped and republished it). I presume that will stay the same b/c it's such a specific title phrase (but not because searches a…

I am skeptical that quotes really work like the plus operator used to work.

For example, try searching the following string in quotes: "the SERP should stay the same".

https://www.google.com/search?q=%22the%20SERP%20should%20sta...

Now, the logical presumption, assuming Google works as people say it does, is that each result will contain that exact string. If no results contain the string, then you should receive no results.

However, for me, results are returned. Did each of the results from that search contain this exact string? For me, they did not.

Re: Dorking: the use of search engines to find very specific data

#129

Post by twelvechairs »

GordonS wrote: post #75
chris_f wrote: post #25

A few corrections: The + (formerly used to force a term to be present in the result) and ~ (also find synonyms) operators have been deprecated. Google now advises to wrap the word in quotes instead of using the +. Google will also automatically look for synonyms without the use of ~. I have seen 'AROUND(n)' mentioned in many other places working as a proximity operator in Google, but I don't believe that is true and…

Worth mentioning that even if you put a term in double quotes, Google still tries to be too clever - you are not guaranteed to get results that contain your quotes search term :/

I think you use 'intext:"search term"' now don't you? Unless that has changed now too

Re: Dorking: the use of search engines to find very specific data

#130

Post by aspenmayer »

solarist wrote: post #110
GordonS wrote: post #106

Earlier quoted context omitted.

AFAICT, the verbatim option gives the same results as if I'd quoted my search term?

In my experience it depends on the number of results, and the results are more accurate with verbatim.

It’s like being able to touch your nose in the dark, or even having been blind since birth.

Verbatim searches are for those easy tasks when you know literally, exactly, what you’re looking for.

Something tells me that a lot of folks on HN are being let down by Google in this area. I don’t mean it in a bad way, I want to identify these issues to help everyone. I wish the UI of search wasn’t so textually-bound.

Is there such a thing as an augmented reality search engine? You search like you do at the library, or when you lost your keys. But for data and virtual objects, they lose out - their evanescence creates a perception gap. There is no correspondence with physical objects, etc. We are getting better with haptics, however.

Rainbows End[0] is a future society I’d choose to live in, especially right about now.

Anyone have any good book recommendations?

[0] Vernor Vinge

https://en.wikipedia.org/wiki/Rainbows_End

Post reply on HN