Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cold Showers: For when people get too hyped up about things

github.com

Post reply on HN
Print view View original

41–50 of 243 posts

Re: Cold Showers: For when people get too hyped up about things

#41

Post by mbrock »

pydry wrote: post #34
carlmr wrote: post #12

Earlier quoted context omitted.

I think the biggest issue with formal verification, is that you need to rewrite the important parts of your code in (for example) TLA+. If it's integrated into the language, like ADA Spark, you don't need to learn so much additional syntax or rewrite parts of your codebase in a language you rarely use (given that you already work in ADA).

My biggest issue with formal verification after doing it a couple of times was how absurdly complex the specification needed to be for it to work. If the spec is 5x more complicated than the code would be then I'm not sure I see much of a point coz you're just creating different spaces for bugs to hide in.

The aim is to have a spec that is much LESS complex than the code, written at a higher level, abstracting away details. If the spec is 5x more complex than the code then indeed there’s no point.

Re: Cold Showers: For when people get too hyped up about things

#42

Post by bob1029 »

Multicomp wrote: post #16

I love that sqlite article. It seems like "everyone" is certain that sqlite can only be used for up to a single query per second, anything more and you need to spin up a triple sharded postgres or Hadoop cluster because it 'needs to scale'. I love being able to show that study, if you properly architect your sqlite system and am willing to purchase hardware, you can go a long long way, much further than almost all co…

SQLite is incredible. If you are struggling to beat the "one query per second" meme, try the following 2 things:

1. Only use a single connection for all access. Open the database one time at startup. SQLite operates in serialized mode by default, so the only time you need to lock is when you are trying to obtain the LastInsertRowId or perform explicit transactions across multiple rows. Trying to use the one connection per query approach with SQLite is going to end very badly.

2. Execute one time against a fresh DB: PRAGMA journal_mode=WAL

If at this point you are still finding that SQLite is not as fast or faster than SQL Server, MySQL, et.al., then I would be very surprised.

I do not think you can persist a row to disk faster with any other traditional SQL technology. SQLite has the lowest access latency that I am aware of. Something about it living in the same process as your business application seems to help a lot.

We support hundreds of simultaneous users in production with 1-10 megs of business state tracked per user in a single SQLite database. It runs fantastically.

Re: Cold Showers: For when people get too hyped up about things

#43

Post by thomascgalvin »

Multicomp wrote: post #16

I love that sqlite article. It seems like "everyone" is certain that sqlite can only be used for up to a single query per second, anything more and you need to spin up a triple sharded postgres or Hadoop cluster because it 'needs to scale'. I love being able to show that study, if you properly architect your sqlite system and am willing to purchase hardware, you can go a long long way, much further than almost all co…

To me, this is the most important line in the article:

> SQLite scales almost perfectly for parallel read performance (with a little work)

They aren't using stock SQLite, they're using SQLite wrapped in Bedrock[1], and their use case is primarily read-only.

SQLite is fantastic at read-only, or read-mostly, use cases. You start to run into trouble when you want to do concurrent writes, however. I tried to use SQLite as the backend of a service a couple of years ago, and it locked up at somewhere around tens of writes per second.

[1]: www.bedrockdb.com

Re: Cold Showers: For when people get too hyped up about things

#44

Post by acdha »

tutfbhuf wrote: post #24
Multicomp wrote: post #16

I love that sqlite article. It seems like "everyone" is certain that sqlite can only be used for up to a single query per second, anything more and you need to spin up a triple sharded postgres or Hadoop cluster because it 'needs to scale'. I love being able to show that study, if you properly architect your sqlite system and am willing to purchase hardware, you can go a long long way, much further than almost all co…

Why choose sqlite over MySQL on a single server (e.g. small vm instance)?

More standard without all of the correctness foot-guns, less to configure and operate, and it’s usually faster. MySQL has to do a lot of work running as a separate process, handling connections, etc. whereas SQLite is just doing file I/O in your current process.

Re: Cold Showers: For when people get too hyped up about things

#45

Post by hans_castorp »

bob1029 wrote: post #42
Multicomp wrote: post #16

I love that sqlite article. It seems like "everyone" is certain that sqlite can only be used for up to a single query per second, anything more and you need to spin up a triple sharded postgres or Hadoop cluster because it 'needs to scale'. I love being able to show that study, if you properly architect your sqlite system and am willing to purchase hardware, you can go a long long way, much further than almost all co…

SQLite is incredible. If you are struggling to beat the "one query per second" meme, try the following 2 things: 1. Only use a single connection for all access. Open the database one time at startup. SQLite operates in serialized mode by default, so the only time you need to lock is when you are trying to obtain the LastInsertRowId or perform explicit transactions across multiple rows. Trying to use the one connectio…

> If at this point you are still finding that SQLite is not as fast or faster than SQL Server, MySQL, et.al., then I would be very surprised.

What about large aggregation queries, that are parallelized by modern DBMS?

Does it still scale that well if you have many concurrent read and write transactions (e.g. on the same table)?

Re: Cold Showers: For when people get too hyped up about things

#46

Post by acdha »

ravenstine wrote: post #39

I think the best takeaway from this is that the software industry makes lots of claims about development processes, but so little actual research is done in trying to validate those processes. It's all mostly based on opinion.

That’s the interesting take for me, too: we love to talk about it as engineering or science but there’s a fairly good argument that this is more aspirational than real in many cases.

Re: Cold Showers: For when people get too hyped up about things

#47

Post by [deleted] »

pixelrevision wrote: post #36
GlennS wrote: post #23

Earlier quoted context omitted.

People who like static typing seem to really like static typing. I'm honestly not convinced it helps that much. And it seems to cost a lot to me. I like database and API schemas though. And I like clojure.spec and function preconditions a lot.

Clojure spec seems like the way to go. I really like that it defines what should be going in and out while leaving it really easy to merge incoming data without having to write a bunch of extra code.

[deleted]

Re: Cold Showers: For when people get too hyped up about things

#48

Post by GlennS »

pantaloony wrote: post #30
GlennS wrote: post #23

Earlier quoted context omitted.

People who like static typing seem to really like static typing. I'm honestly not convinced it helps that much. And it seems to cost a lot to me. I like database and API schemas though. And I like clojure.spec and function preconditions a lot.

I don’t get the cost claims. The time it takes to note which type I intend something to be is mostly either so low that I recover it via improved hints and such very quickly, or larger but only because I’m documenting something complex enough that I should have documented it anyway, whether or not I was using static types, because it’ll be hell for other people or future-me to figure out otherwise. It seems like a la…

I would like to be able to explain the cost part better. It may just be personal bias of course.

1. There's no guarantee the correct theoretical model of your program fits the type system of your programming language.

2. Sometimes there are multiple correct models for different purposes in the same program, similar to how sometimes you need multiple views onto the same database tables.

3. Sometimes you just need the ability to bodge things.

Re: Cold Showers: For when people get too hyped up about things

#49

Post by simiones »

pantaloony wrote: post #30
GlennS wrote: post #23

Earlier quoted context omitted.

People who like static typing seem to really like static typing. I'm honestly not convinced it helps that much. And it seems to cost a lot to me. I like database and API schemas though. And I like clojure.spec and function preconditions a lot.

I don’t get the cost claims. The time it takes to note which type I intend something to be is mostly either so low that I recover it via improved hints and such very quickly, or larger but only because I’m documenting something complex enough that I should have documented it anyway, whether or not I was using static types, because it’ll be hell for other people or future-me to figure out otherwise. It seems like a la…

I think the general feeling is that there are some code patterns that are safe and easy to do with dynamic typing, but impossible with simple type systems or more complex with more advanced type system.

An example would be Common Lisp's `map` function [0] (it takes a number of sequences and a function that has as many parameters as there are sequences). It would be hard to come up with a type for this in Java, and it would be a pretty complicated type in Haskell.

Another example of many people's experience with static typing is the Go style of language, where you can't write any code that works for both a list of strings and a list of numbers. This is no longer common, but it used to be very common ~10-15 years ago and many may have not looked back.

[0] http://www.lispworks.com/documentation/HyperSpec/Body/f_map....

Re: Cold Showers: For when people get too hyped up about things

#50

Post by ahelwer »

jedimastert wrote: post #35
ahelwer wrote: post #11

Earlier quoted context omitted.

Yes. Like stoicism. You aren't emotionally insensitive/unintelligent, you're a masculine Roman general!

How is stoicism a SV thing? It's kinda been around for a while...

Everyone I know who identifies as a stoic is an emotionally stunted software engineer who realistically isn't tasked with stoically shouldering very much of anything.
Post reply on HN