Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Fastest search engine in the world

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 47 posts

Re: Show HN: Fastest search engine in the world

#31

Post by marcuslager »

softwaredoug wrote: post #29

> Diverting people from using Lucene/Elasticsearch would then be a breeze. Maybe. But most people's problems with Lucene/Elasticsearch aren't really speed. They use Lucene because its feature rich and been worked on for almost two decades. And there's plenty of strategies to mitigate any speed problems (caching, sharding, etc) with lots of knowledge spread throughout hundreds of orgs to the point where "making Elasti…

I'm glad I posted here because one point is maybe finally clear to me about speed. It's not a good word for performance. It's the wrong word to use.

The great folks at Elasticsearch would _love_ for Lucene to be more performant. It would make life so much easier for them.

The Lucene team spend a good buck on their nightly performance tests. It's astonishing how well-tested Lucene is.

I wonder why I'm faster at writing and reading. Maybe it's because I have been benchmarking against an older version (4.8). But still. I wonder if my tests are all wrong or if I just got lucky in my design. ResinDB has flaws. It puts massive pressure on GC at writing time, if your batches are huge. I'm working hard at optimizing that achilees heel away. Before I have completely done so, writing speed is achieved through lots of memory allocations. It's surprisingly easy though to move away from using GC as a service.

But performance is not a feature anymore?

Edit:typos and tried to clarify things

Re: Show HN: Fastest search engine in the world

#32

Post by marcuslager »

CyberDildonics wrote: post #30
marcuslager wrote: post #15

Earlier quoted context omitted.

My claims are backed up by the code I've spent blood and sweat to create. Disprove me please because I need to know of scenarios that I need to solve that goes into vNext, scenarios were I'm currently not doing great. Edit: and also: I'm reaching out to you guys not because I want a pat on the back or free PR. I'm looking for advise as to how to move from having unique tech to having a business. Is this the right for…

> having unique tech You have claims and they aren't even unique. > Disprove me please That's not how it works. Why would I waste my time testing your software when you don't seem to possess common sense or experience? The probability that you can back up what you say is excessively low.

I'm scrolling up to see where you hit a nerve with me (or is it the other way around?). Anyway, sorry about that.

Re: Show HN: Fastest search engine in the world

#33

Post by [deleted] »

marcuslager wrote: post #15
CyberDildonics wrote: post #13

Earlier quoted context omitted.

Back up you claims. Have benchmarks and demos. Why say these things when there are neighther benchmarks or demos out there?

My claims are backed up by the code I've spent blood and sweat to create. Disprove me please because I need to know of scenarios that I need to solve that goes into vNext, scenarios were I'm currently not doing great. Edit: and also: I'm reaching out to you guys not because I want a pat on the back or free PR. I'm looking for advise as to how to move from having unique tech to having a business. Is this the right for…

[deleted]

Re: Show HN: Fastest search engine in the world

#34

Post by visarga »

marcuslager wrote: post #12

Thx for the feedback. What I think you should and hope you already do realise is Lucene is nowhere near maximum performance for full-text search nor is it's relevance. And implementing new scoring routines is a drag in Lucene. Google is also nowhere near maximum relevance. I like word2vec. That model fits into my world view. I'm going to implement it and then take it further. Hopefully while being funded. If not then…

If you want to do word vector similarity search, try the "annoy" library from Spotify. It's much much faster than Gensim. https://github.com/spotify/annoy

Re: Show HN: Fastest search engine in the world

#38

Post by amirouche »

marcuslager wrote: post #37
amirouche wrote: post #36

FWIW, various HNers expressed the interest of having something similar to algolia service. Basically, some service than can index a website (including static websites) and provide a search API for it. Good luck!

Stolen!

What do you mean?

Re: Show HN: Fastest search engine in the world

#40

Post by z3t4 »

marcuslager wrote: post #27
z3t4 wrote: post #25

Earlier quoted context omitted.

i dont even know what you are selling. show me a demonstration. can i run it in nodejs ? or sell directly to those who know exacly what you are talking about and solves their particular need.

I agree that a library such as this project is not at all as consumer-friendly as an application is. Some might even call it completely unsexy. It's a component of something bigger though, something you can indeed call into from nodejs. But that's another project. >or sell directly to those who know exacly what you are talking about Yeah I've been thinking I should try to get a few gigs as a speaker at tech meetups o…

Thank you, I think I somewhat understand now. For it to be accessible for me, I'm used to something like:

installation:

  npm install resin
usage:

  var resin = require("resin");
  var wikipedia = resin.init({file: "c:\temp\wikipedia.json", dir: "c:\resin\data\wikipedia"});
  var dogs = wikipedia.query("title: dog");
  
  // or ...
  var players = resin.init({dir: "c:\resin\data\playerData"});
  var oldPlayers = players.query("age > 30");
Post reply on HN