Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

beets.io

Post reply on HN
Print view View original

91–100 of 149 posts

Re: We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

#91

Post by greenleafjacob »

omarforgotpwd wrote: post #16

For any database that isn't huge, a library embedded into your application is going to be faster than anything that has to communicate with a server over a socket connection. Though both execute SQL queries, SQLite is completely different than relational database servers and appropriate many places where running a full RDBMS is not. For example, you can't run MySQL or Postgres on the iPhone, but you can use SQLite.

I've found SQLite absolutely amazing for getting the power of SQL for R data frames.

Re: We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

#92

Post by IndianAstronaut »

greenleafjacob wrote: post #91
omarforgotpwd wrote: post #16

For any database that isn't huge, a library embedded into your application is going to be faster than anything that has to communicate with a server over a socket connection. Though both execute SQL queries, SQLite is completely different than relational database servers and appropriate many places where running a full RDBMS is not. For example, you can't run MySQL or Postgres on the iPhone, but you can use SQLite.

I've found SQLite absolutely amazing for getting the power of SQL for R data frames.

The set up is quite interesting. The sql data frames library will embed a dataframe in sqllite and let you use sql. It has been quite useful in transitioning our SQL data analysts to R.

Re: We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

#93

Post by [deleted] »

greenleafjacob wrote: post #91
omarforgotpwd wrote: post #16

For any database that isn't huge, a library embedded into your application is going to be faster than anything that has to communicate with a server over a socket connection. Though both execute SQL queries, SQLite is completely different than relational database servers and appropriate many places where running a full RDBMS is not. For example, you can't run MySQL or Postgres on the iPhone, but you can use SQLite.

I've found SQLite absolutely amazing for getting the power of SQL for R data frames.

[deleted]

Re: We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

#94

Post by int_19h »

greenleafjacob wrote: post #91
omarforgotpwd wrote: post #16

For any database that isn't huge, a library embedded into your application is going to be faster than anything that has to communicate with a server over a socket connection. Though both execute SQL queries, SQLite is completely different than relational database servers and appropriate many places where running a full RDBMS is not. For example, you can't run MySQL or Postgres on the iPhone, but you can use SQLite.

I've found SQLite absolutely amazing for getting the power of SQL for R data frames.

Could you expand on some of your use cases? An R data frame, by definition, has to fit into memory, so it would seem that any sort of map/filter/group/fold operation would be fastest if performed in-memory, as well. And I assumed that e.g. joining data frames (where you would run out of memory really quickly if your datasets are large to begin with) would be uncommon... am I wrong?

Re: We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

#95

Post by int_19h »

While we're speaking of SQLite; one thing that has little exposure that could probably use more is that it now ships with Windows as a system DLL:

https://engineering.microsoft.com/2015/10/29/sqlite-in-windo...

Between that, and packages readily available on most Linux and BSD distros out there (and, in most cases, installed by default), it's well on its way to become a de facto standard system API for relational storage.

Re: We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

#96

Post by pilif »

chjj wrote: post #81
orf wrote: post #58

Earlier quoted context omitted.

> What took 300-400ms in leveldb took 1.6 seconds in postgres (on the repl. in my actual node it would have taken longer due to deserialization of the utxos). What took 1.6 seconds in postgres took over 30 seconds in SQlite. I'm sorry to say but you're almost certainly doing something wrong then.

It's possible, but it was a very simple database layout. Pretty hard to screw up. Then again, I screw up simple things all the time. But whatever I screwed up would likely have been duplicated on the postgres side since the sqlite and postgres setups were nearly identical. Postgres performed fine compared to sqlite. It was a bit behind leveldb, but I figure the overhead of flushing the data to a socket has an added c…

`select * from utxos where address =?` should not take 300ms no matter the size or type of database (assuming there is an index on address).

Re: We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

#98

Post by eru »

mianos wrote: post #87
SwellJoe wrote: post #21

I never stop being impressed at how often people will jump to odd, unsupportable, conclusions like, "using MySQL will make this thing faster". I've seen it so many times over the years regarding users and email configurations. I can't count the number of times I've dropped into someone's badly behaving mail configuration and found they had MySQL hosting the users, and explained it was for "performance" reasons. Someh…

So many times I see people handing out a root password to edit /etc/password. Sure, it may be crazy for performance but, even a trivial number of users and aliases are more easily managed with a remote mysql client. Everyone that has used postfix in a production environment knows the pain otherwise, the rest feel free to vote down.

> So many times I see people handing out a root password to edit /etc/password.

Shouldn't user groups give a bit more accountability there?

Re: We’re pretty happy with SQLite and not urgently interested in a fancier DBMS

#99

Post by kefka_p »

mgkimsal wrote: post #86
kefka_p wrote: post #65

Some people are unfamiliar with the phrase "right tool for the job". As the developers behind the project, I'd have to think the authors are in the best position to make the determination about which tool is appropriate.

"right tool for the job" ends up being almost meaningless unless the players involved have enough experience with multiple tools to make a useful judgement. Too often "right tool" is "whatever I've already used before". :/

That is true. I failed to adjust my perspective for the fact that not everyone has nearly thirty years of programming experience given I still consider myself an amateur.

Still we should recall the utility gained in using the right tool is finite and variable. In some cases can be like night and day while in other cases a moderate or negligble improvement might be all you see. Under some circumstances familiarity with a given tool may outweigh the advantages of using a better suited tool - especially if a team lacks well-rounded expertise.

Reinforcing the other side of the argument is the fact that programmers are often subject to constraints placed on them from on high that restrict the choices they might make in such.

Given the specific situation, the author seems to build a pretty good case for SQLite at least by my own limited understanding of the facts.

Post reply on HN