Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cold Showers: For when people get too hyped up about things

github.com

Post reply on HN
Print view View original

131–140 of 243 posts

Re: Cold Showers: For when people get too hyped up about things

#131

Post by albntomat0 »

protonfish wrote: post #100
nafey wrote: post #29

Earlier quoted context omitted.

I think it gives people a sense of satisfaction in modeling real world in the relations between classes. The assertion seems to be that if to solve a problem it has to be correctly modelled into the type system of the language. Once the modelling is done correctly solution will arise by itself. On the other end people who prefer weakly typed languages see problems as primarily that of data transformation. For example…

Please don't use weak/strong to denote type systems. Those terms are highly subjective and even non-technical people would quickly form an opinion about which is better. (Strong is good, weak is bad.) Static/dynamic is more accurate and less opinionated terminology.

They are not the same, at least according to the definitions I'm familiar with.

Static/dynamic is whether type checking is done at compile time or run type.

Strong/weak is how flexible the language is with type conversion.

Another explanation: https://en.hexlet.io/courses/intro_to_programming/lessons/ty...

Re: Cold Showers: For when people get too hyped up about things

#132

Post by barrkel »

bob1029 wrote: post #42
Multicomp wrote: post #16

I love that sqlite article. It seems like "everyone" is certain that sqlite can only be used for up to a single query per second, anything more and you need to spin up a triple sharded postgres or Hadoop cluster because it 'needs to scale'. I love being able to show that study, if you properly architect your sqlite system and am willing to purchase hardware, you can go a long long way, much further than almost all co…

SQLite is incredible. If you are struggling to beat the "one query per second" meme, try the following 2 things: 1. Only use a single connection for all access. Open the database one time at startup. SQLite operates in serialized mode by default, so the only time you need to lock is when you are trying to obtain the LastInsertRowId or perform explicit transactions across multiple rows. Trying to use the one connectio…

You've just moved a bunch of problems to whatever is accessing SQLite. How do you scale out the application server compute if it needs to make transactionally conditional updates?

E.g.:

    transaction {
      if (expensiveFunction(query()))
        update();
    }
(My applications always get much faster when I plug them into Postgres after SQLite. But then I do do the odd sort and group by, not OLAP, but because they express the computation I want and Postgres is simply much better at that.)

Re: Cold Showers: For when people get too hyped up about things

#133

Post by archarios »

quonn wrote: post #10

I wish it would be possible to have better studies for that. I believe that static typing has huge benefits as software scales. I also believe that the type system of TypeScript is actually stronger in practice than the Java or C# one (despite theoretical weaknesses). It has the right tradeoffs (e.g. structural equivalence, being able to type strings, being able to check that all cases are handled, etc.) It would be…

I've contributed to large codebases that have static typing (C++ and TypeScript) and dynamic typing (JavaScript) and I've come to the conclusion over the years that static typing isn't really worth it as long as you have the discipline to write tests for your code. The most basic unit tests cover type checking concerns. Refactoring might require a bit more search/replace but I don't see how that is a big deal. Tests make refactoring safer than with just types. Tests act as good documentation of how you expect your code to behave and what expected inputs/outputs there are. You just don't get autocomplete which is a pretty overrated feature imo.

Re: Cold Showers: For when people get too hyped up about things

#134

Post by jdm2212 »

archarios wrote: post #128
jdm2212 wrote: post #77

Earlier quoted context omitted.

The study IMHO is reality. Every large, actively maintained system I've ever heard of uses a strongly typed language, or was written in a dynamically typed language but has converted to a gradually typed language. You can't safely refactor without compile time type checking, and you can't maintain a non-trivial system over the long-term if you can't refactor it.

Just write tests and suddenly refactoring is possible without static typing. Those tests will also cover type checking concerns implicitly.

If you add a new value to an enum, or a new argument to a method, you have to know all the places where you do switches on the enum or call the method, which is hard to do without static types.

Re: Cold Showers: For when people get too hyped up about things

#135

Post by barrkel »

tristor wrote: post #103
castorp wrote: post #86

Earlier quoted context omitted.

> At the end of the day, you still have to go to disk on writes, and this must be serialized against reads for basic consistency reasons. No, absolutely not. That's why modern databases use a thing called multi version concurrency control. You can run (multiple) queries on the same table that is updated by multiple transactions at the same time without one blocking the others (assuming the write transactions don't bl…

"serialized" here doesn't really mean processed in serial, it means "serializable" in the context of database information theory. Databases have special concurrency control requirements in order to create hard guarantees on database consistency. You can process queries in parallel and still have a serializable result, because of transaction coordination. Doing this on one server is much easier than doing this across…

There's an enormous leap between something which is slow on SQLite and something which requires etl into a data warehouse or similar tech, columnar store etc.

I mean, at least three orders of magnitude, minimum.

It's just a ludicrous argument. SQLite is fine for a file format, and in very specific dumb CRUD scenarios it's just about ok. But it's not worth sticking with if you need anything interesting over any volume of data, far far far below what would warrant a different DB tech to rdbms.

Ironically, you're more representative of the "we'll need Hadoop" crowd.

Re: Cold Showers: For when people get too hyped up about things

#136

Post by nafey »

albntomat0 wrote: post #131
protonfish wrote: post #100

Earlier quoted context omitted.

Please don't use weak/strong to denote type systems. Those terms are highly subjective and even non-technical people would quickly form an opinion about which is better. (Strong is good, weak is bad.) Static/dynamic is more accurate and less opinionated terminology.

They are not the same, at least according to the definitions I'm familiar with. Static/dynamic is whether type checking is done at compile time or run type. Strong/weak is how flexible the language is with type conversion. Another explanation: https://en.hexlet.io/courses/intro_to_programming/lessons/ty...

Are these dimensions orthogonal though? For example is there any Strongly typed dynamic language?

Re: Cold Showers: For when people get too hyped up about things

#137

Post by WrtCdEvrydy »

saberience wrote: post #76
bob1029 wrote: post #42

Earlier quoted context omitted.

SQLite is incredible. If you are struggling to beat the "one query per second" meme, try the following 2 things: 1. Only use a single connection for all access. Open the database one time at startup. SQLite operates in serialized mode by default, so the only time you need to lock is when you are trying to obtain the LastInsertRowId or perform explicit transactions across multiple rows. Trying to use the one connectio…

Are you living on another planet? Are you seriously suggesting people replace SqlServer or MySQL with SQLite? If you can come to my company and replace our 96-core SqlServer boxes with SQLite I'll pay you any salary you ask for.

> come to my company and replace our 96-core SqlServer boxes with SQLite I'll pay you any salary you ask for.

I also had a server with 96 cores until we realized a developer had inadvertently made a query happen every time a scroll event fired... it was a nice chunk of chance saved.

Re: Cold Showers: For when people get too hyped up about things

#138

Post by WrtCdEvrydy »

quickthrower2 wrote: post #40
Multicomp wrote: post #16

I love that sqlite article. It seems like "everyone" is certain that sqlite can only be used for up to a single query per second, anything more and you need to spin up a triple sharded postgres or Hadoop cluster because it 'needs to scale'. I love being able to show that study, if you properly architect your sqlite system and am willing to purchase hardware, you can go a long long way, much further than almost all co…

I don’t use SQLite on a server for the simple reason I’m to lazy to look after it. There are cloud managed sql server or psql databases. AWS backs me up by default. Why mess around with SQLite. Not knocking SQLite - great for desktop apps or maybe local dev environments.

It depends on whether your data is confidential as well... I have been known to send a full SQLite dump to a small app that needed a lot of local data.

You can store it in localStorage and read it making your reload be a lot smaller. If it's not in localStorage, you can just request a fresh copy of SQLite from us.

Re: Cold Showers: For when people get too hyped up about things

#139

Post by realtalk_sp »

fmakunbound wrote: post #61
rwoerz wrote: post #28

We software engineers are still more like alchimists rather than chemists. That list reminds me of [1], which rants about this state of affairs and [2] that puts many beliefs to the test. [1] https://youtu.be/WELBnE33dpY [2] https://www.oreilly.com/library/view/making-software/9780596...

In addition to that, I also feel calling ourselves engineers is a stretch.

I suspect many developers would find software a lot less appealing if it were closer to a traditional engineering discipline (slow, unforgiving, dramatically reduced expressive power and creative potential, etc).

I'm somewhat ADD and get bored easily, so not only do I need to do something more like software but I also have to stay as broad and high-level as possible within the discipline, to stave off ennui. NOTE: very much not arguing this is a good way to go through life.

Re: Cold Showers: For when people get too hyped up about things

#140

Post by bob1029 »

mumblemumble wrote: post #113
bob1029 wrote: post #71

Earlier quoted context omitted.

We aren't running any reports on our databases like this. I would argue it is a bad practice in general to mix OLTP and OLAP workloads on a single database instance, regardless of the specific technology involved. If we wanted to run an aggregate that could potentially impact live transactions, we would just copy the SQLite db to another server and perform the analysis there. We have some telemetry services which ope…

> I am not aware of any hosted SQL technology which is capable of magically interleaving large aggregate queries with live transactions and not having one or both impacted in some way. At the end of the day, you still have to go to disk on writes, and this must be serialized against reads for basic consistency reasons. I do sometimes wonder if dirty reads are what the business folks actually want. Not necessarily unc…

We effectively do dirty reads when we just go around and copy the databases for purposes of offline analysis. Our use cases for aggregation queries are all tolerant to incomplete recent data. Most of the time we are running analysis 24 hours after whatever event, so we never get into a situation where we can't answer important business questions because data hasn't been flushed to disk yet.

The fact that most of the stuff we care about is time-domain & causal means that we can typically leverage this basic ideology. Very rarely does a time-series aggregate query need to be consistent with the OLTP workloads in order to satisfy a business question.

Post reply on HN