Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A search engine that favors text-heavy sites and punishes modern web design

search.marginalia.nu

Post reply on HN
Print view View original

631–640 of 735 posts

Re: A search engine that favors text-heavy sites and punishes modern web design

#631

Post by mananaysiempre »

leephillips wrote: post #593
mananaysiempre wrote: post #584

Earlier quoted context omitted.

In English, maybe; in Russian, I frequently find myself reaching for the nonexistent “morphology but not synonyms” operator (as the same noun phrase can take a different form depending on whether it is the subject or the object of a verb, or even on which verb it is the object of); even German should have the same problem AFAIU, if a bit milder. I don’t dare think about how speakers of agglunative languages (Finnish,…

Ah, OK. I don’t know anything about Russian. This is a hard problem. I think the solution is something like what you suggest: more operators allowing different transformations. Even in English, I would like a "you may pluralize but nothing else" operator.

Well it’s not that alien, it (along with the other Eastern Slavic languages, Ukrainian and Belarusian) is mostly a run-of-the-mill European language (unlike Finnish, Estonian or Hungarian) except it didn’t lose the Indo-European noun case system like most but instead developed even more cases. That is, where English or French would differentiate the roles of different arguments of a verb by prepositions or implicitly by position, Russian (like German and Latin) has a special axis of noun forms called “case” which it uses for that (and also prepositions, which now require a certain case as well—a noun form can’t not have a case like it can’t not have a number).

There are six of them (nominal [subject], genitive [belonging, part, absence, “of”], dative [indirect object, recipient, “to”], accusative [direct object], instrumental [device, means, “by”], prepositional [what the hell even is this]), so you have (cases) × (numbers) = 6 × 2 = 12 noun forms, and adjectives agree in number and gender with their noun, but (unlike Romance languages) plurals don’t have gender, so you have (cases) × (numbers and genders) = 6 × (3 + 1) = 24 adjective forms.

None of this would be particularly problematic, except these forms work like French or Spanish verbs: they are synthetic (case, number and gender are all a single fused ending, not orthogonal ones) and highly convoluted with a lot of irregularities. And nouns and adjectives are usually more important for a web search than verbs.

Re: A search engine that favors text-heavy sites and punishes modern web design

#632

Post by mananaysiempre »

dahauns wrote: post #583
mananaysiempre wrote: post #547

Earlier quoted context omitted.

I’m not against a stemmer, actually, just against the aggressive concordances (?) that Google now employs, like when it shows me X in Banach spaces (the classical, textbook case) when I’m specifically searching for X in Fréchet spaces (the generalization I want to find but am not sure exists); of course Banach spaces and Fréchet spaces are almost exclusively encountered in the same context, but it doesn’t mean that o…

English is more the outlier in regard to Germanic languages, try German or Finnish, with their wonderful compounds :) https://e.humanities.uva.nl/publications/2004/kamp_lang04.pd...

Well yeah, English is kind of weird, but Finnish isn’t a Germanic language at all? It’s not even Indo-European, so even Hindi is ostensibly closer to English than Finnish. I understand Standard German (along with Icelandic) is itself a bit atypical in that it hasn’t lost its cases when most other Germanic languages did.

Re compounds, I expected they would be more or less easy to deal with by relatively dumb splitting, similar to greedy solutions to the “no spaces” problem of Chinese and Japanese, and your link seems to bear that out. But yeah, cheers to more language-specific stuff in your indexing. /s

Re: A search engine that favors text-heavy sites and punishes modern web design

#633

Post by karlzt »

marginalia_nu wrote: post #3

Yeah so this is my project. It's very much a work in progress, but occasionally I think it works remarkably well for something I cobbled together alone out of consumer hardware and home-made code :-)

I can't see the letters in the disturbing white search box, I'm on duckduckgo, brave, monocles, jquarks, smartcookieweb on Android.

Re: A search engine that favors text-heavy sites and punishes modern web design

#635

Post by wolverine876 »

marginalia_nu wrote: post #624
wolverine876 wrote: post #614

Earlier quoted context omitted.

www.ft.com gets 'no such domain'

I added it now, but it turns out it's behind a CDN so I still can't crawl it.

Thanks for responding and especially thanks for the search engine. What a breath of fresh air, and access, it feels like, to real people.

Re: A search engine that favors text-heavy sites and punishes modern web design

#636

Post by QuibbleQuota »

andrepd wrote: post #457
echelon wrote: post #456

Earlier quoted context omitted.

The new evergreen is refreshed sludge for bottom dollar. College kids stealing Reddit comments or moving around paragraphs from old articles. Or linking to linked blogs that link elsewhere. It's all stamped with Google Ads, of course, and then Google ranks these pages high enough to rake in eyeballs and ad dollars. Also there's the fact that each year, the average webpage picks up two more video elements / ad players…

Really makes you wonder: you play whack a mole and tackle the symptoms with initiatives like this search engine. But the root of that problem and many many others is the same: advertising. Why don't we try to tackle that?

Let’s go a few levels deeper and question our consumption culture

Re: A search engine that favors text-heavy sites and punishes modern web design

#637

Post by post_below »

dvogel wrote: post #356
amenod wrote: post #350

Earlier quoted context omitted.

> is that the algorithms prioritize newer pages over older pages. They do? That would explain a lot - but ironically, I can't find a good source on this. Do you have one at hand?

It is pretty obvious if you search for any old topic that is also covered incessantly by the news. "royal family" is a good example. There's no way those news stories published an hour ago are listed first due to a high PageRank score (which necessarily depends on time to accumulate inbound links).

There are of course a variety of factors, including the popularity of the site the page is published on. The signals related to the site are often as important as the content on the page itself. Even different parts of the same site can lend varying weight to something published in that section.

Engagement, as measured in clicks and time spent on page, plays a big part.

But you're right, to a degree, as frequently updated pages can rank higher in many areas. A newly published page has been recently updated.

A lot depends on the (algorithmically perceived) topic too. Where news is concerned, you're completely right, algos are always going to favor newer content unless your search terms specify otherwise.

PageRank, in it's original form, is long dead. Inbound link related signals are much more complex and contextual now, and other types of signals get more weight.

Re: A search engine that favors text-heavy sites and punishes modern web design

#638

Post by 1vuio0pswjnm7 »

I had an idea for an alternative search engine a few years ago thats a bit simpler to implement than this one. First we extract all the external links from Wikipedia dumps. Then we ingest only those sites into the index. The entire database comprises sites that have already been screened by Wikipedians. Gaming Wikipedia is generally more difficult than gaming Google or Bing. In theory, SERPs would be of a less commercial and more substantive nature than those we get from Google and Bing.

Re: A search engine that favors text-heavy sites and punishes modern web design

#639

Post by handelaar »

allknowingfrog wrote: post #211
fouric wrote: post #168

Earlier quoted context omitted.

The project explicitly bills itself as a "search engine", not an "interesting and unexpected material surfacer". Moreover, projecting emotions like "angry" onto a comment in order to discredit the content of the comment (hey! is that an ad-hominem?) is just about exactly the opposite of the discussions that the HN mods are trying to curate, and the discussions that I like to see here.

If you click through to the About page, I think you'll see that "interesting and unexpected material surfacer" is a fairly apt description of the project.

I think in fairness that when "interesting and unexpected material surfacer" is merely a euphemism for "we didn't bother indexing the things you might actually be looking for", a degree of scepticism isn't unwarranted.

(Source: I looked up several Irish politicians because I run an all-text website containing every single word that they say in parliament. I got nothing of use, or even of interest, for anything.)

Re: A search engine that favors text-heavy sites and punishes modern web design

#640

Post by kkiinnpptt »

addandsubtract wrote: post #522
MaysonL wrote: post #468

Earlier quoted context omitted.

Google Info - for when you're looking for information, not shopping advice or lists!

Google info? Can you give me a sample query of what you mean?

It was a joke. The joke being that Google should launch at new product called Google Info, that would actually give you information when you search.
Post reply on HN