Live data from Hacker News

Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

rendiment.io

11–20 of 27 posts

Re: Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

#11
for 50,000 rows I'd much rather just use fzf/nucleo/tv against json files instead of dealing with database schemas. When it comes to dealing with embedding vectors rather than plaintext then it gets slightly more annoying but still feels like such an pain in the ass to go full database when really it could still be a bunch of flat open files.

More of a perspective from just trying to index crap on my own machine vs building a SaaS

Re: Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

#14
post #8

these days i find myself yearning to type "Beatles abbey rd" and find only "Beatles abbey rd"

I learned this the hard way on a book platform I'm working on. While semantic search is useful for discovery, we found that prioritizing exact matches is critical. It seems users get pretty frustrated if they type a specific title and get a list of conceptually similar results instead of the actual book. We ended up having to tune the ranking to heavily favor literal string matches over the vector distance to keep people from bouncing.

Re: Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

#15
post #8

these days i find myself yearning to type "Beatles abbey rd" and find only "Beatles abbey rd"

I learned this the hard way on a book platform I'm working on. While semantic search is useful for discovery, we found that prioritizing exact matches is critical. It seems users get pretty frustrated if they type a specific title and get a list of conceptually similar results instead of the actual book. We ended up having to tune the ranking to heavily favor literal string matches over the vector distance to keep pe…

everything you are saying rings perfectly true to me but there's an additional problem I encounter. (i'm going to make up my example because i'm lazy to check but you'll get the idea) say you want to look up "Alexander the Great"...

...God help you if Brad Pitt and or the Jonas Brothers ever played a role with exactly that name-match. The web and search (and the culture?) have become super biased toward video especially commercial offerings, and the sorting ranked by popularity means pages and pages of virtually identical content about that which you are not interested in.

Re: Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

#16
post #15

Earlier quoted context omitted.

I learned this the hard way on a book platform I'm working on. While semantic search is useful for discovery, we found that prioritizing exact matches is critical. It seems users get pretty frustrated if they type a specific title and get a list of conceptually similar results instead of the actual book. We ended up having to tune the ranking to heavily favor literal string matches over the vector distance to keep pe…

everything you are saying rings perfectly true to me but there's an additional problem I encounter. (i'm going to make up my example because i'm lazy to check but you'll get the idea) say you want to look up "Alexander the Great"... ...God help you if Brad Pitt and or the Jonas Brothers ever played a role with exactly that name-match. The web and search (and the culture?) have become super biased toward video especia…

Related but I wish Wikipedia would provide a filter against movies, music, pop culture related topics. They take up a huge amount of the namespace for things for whatever reason and often directs me to unintended pages.

Re: Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

#17
post #16
post #15

Earlier quoted context omitted.

everything you are saying rings perfectly true to me but there's an additional problem I encounter. (i'm going to make up my example because i'm lazy to check but you'll get the idea) say you want to look up "Alexander the Great"... ...God help you if Brad Pitt and or the Jonas Brothers ever played a role with exactly that name-match. The web and search (and the culture?) have become super biased toward video especia…

Related but I wish Wikipedia would provide a filter against movies, music, pop culture related topics. They take up a huge amount of the namespace for things for whatever reason and often directs me to unintended pages.

https://en.wikipedia.org/w/index.php?search=melancholia+-mov...

Re: Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

#18
FWIW, the performance considerations section is a little simplistic, and probably assumes that exact dataset/problem.

For GIN for example, perfomance depends a lot on the size of the search input (the fewer characters, the more rows to compare) as well as the number of rows/size of the index.

It also mentions GiST (another type of index which isn't mentioned anywhere else in the article)..

Re: Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

#19

> Abbey Road > The Dark Side of the Moon > OK Computer Those are my 3 personal records ever. I feel so average now...

The other two are popular but "Dark Side of the Moon" in particular was extremely popular. Like, top 10 albums ever level popular.

Re: Find 'Abbey Road when type 'Beatles abbey rd': Fuzzy/Semantic search in Postgres

#20
post #8

these days i find myself yearning to type "Beatles abbey rd" and find only "Beatles abbey rd"

I learned this the hard way on a book platform I'm working on. While semantic search is useful for discovery, we found that prioritizing exact matches is critical. It seems users get pretty frustrated if they type a specific title and get a list of conceptually similar results instead of the actual book. We ended up having to tune the ranking to heavily favor literal string matches over the vector distance to keep pe…

why did you have to learn this the hard way?
Post reply on HN