Live data from Hacker News

Cold Showers: For when people get too hyped up about things

github.com

221–230 of 243 posts

Re: Cold Showers: For when people get too hyped up about things

#221

Earlier quoted context omitted.

There's an enormous leap between something which is slow on SQLite and something which requires etl into a data warehouse or similar tech, columnar store etc. I mean, at least three orders of magnitude, minimum. It's just a ludicrous argument. SQLite is fine for a file format, and in very specific dumb CRUD scenarios it's just about ok. But it's not worth sticking with if you need anything interesting over any volume…

> Ironically, you're more representative of the "we'll need Hadoop" crowd. That seems an especially odd assertion to make. What is your basis for this comment? I'd suggest it makes clear you know nothing about me.

A defense of SQLite on unsuitability to OLAP grounds, when many aggregate and analytic functions can be performed perfectly acceptably on RDBMSes like Postgres (and even MySQL, with care), diving straight to the append-only, data pipeline with ETL approaches.

Re: Cold Showers: For when people get too hyped up about things

#222
post #79

Earlier quoted context omitted.

I can assure you that I live on the same planet as everyone else posting here. Whether or not I could perform this miracle depends entirely on your specific use cases. Many people who have this sort of reaction are coming from a place where there is heavy use of the vendor lock-in features such as SSIS and stored procedures. If you are ultimately just trying to get structured business data to/from disk in a consisten…

Is SQLite likely to be faster than postgres? In terms of ease of use / admin overhead I consider them mostly equivalent. I thought the main problem with SQLite was it was slow tih concurrent writers. Whereas the "bigger" SQL databases have code that allows concurrent writes.

I think there are a lot of places where sqlite can outperform postgres... This is read-heavy and latency-critical apps, where additional hop is costly, for example.

Re: Cold Showers: For when people get too hyped up about things

#223
post #42

I love that sqlite article. It seems like "everyone" is certain that sqlite can only be used for up to a single query per second, anything more and you need to spin up a triple sharded postgres or Hadoop cluster because it 'needs to scale'. I love being able to show that study, if you properly architect your sqlite system and am willing to purchase hardware, you can go a long long way, much further than almost all co…

SQLite is incredible. If you are struggling to beat the "one query per second" meme, try the following 2 things: 1. Only use a single connection for all access. Open the database one time at startup. SQLite operates in serialized mode by default, so the only time you need to lock is when you are trying to obtain the LastInsertRowId or perform explicit transactions across multiple rows. Trying to use the one connectio…

Why single connection? Sqlite in WAL mode will allow to co-exist writer and readers.

Re: Cold Showers: For when people get too hyped up about things

#224

Earlier quoted context omitted.

I don’t get the cost claims. The time it takes to note which type I intend something to be is mostly either so low that I recover it via improved hints and such very quickly, or larger but only because I’m documenting something complex enough that I should have documented it anyway, whether or not I was using static types, because it’ll be hell for other people or future-me to figure out otherwise. It seems like a la…

I think the general feeling is that there are some code patterns that are safe and easy to do with dynamic typing, but impossible with simple type systems or more complex with more advanced type system. An example would be Common Lisp's `map` function [0] (it takes a number of sequences and a function that has as many parameters as there are sequences). It would be hard to come up with a type for this in Java, and it…

Not that complicated even in C++

template auto map(Func && func, Lists &&... lists) -> std::vector::type::value_type>()...))>;

Re: Cold Showers: For when people get too hyped up about things

#225

Earlier quoted context omitted.

I think the general feeling is that there are some code patterns that are safe and easy to do with dynamic typing, but impossible with simple type systems or more complex with more advanced type system. An example would be Common Lisp's `map` function [0] (it takes a number of sequences and a function that has as many parameters as there are sequences). It would be hard to come up with a type for this in Java, and it…

Not that complicated even in C++ template auto map(Func && func, Lists &&... lists) -> std::vector ::type::value_type>()...))>;

> std::declval::type::value_type>()

Yes, nothing hard to understand or discover about that at all...

Re: Cold Showers: For when people get too hyped up about things

#226
post #68

Earlier quoted context omitted.

I think the general feeling is that there are some code patterns that are safe and easy to do with dynamic typing, but impossible with simple type systems or more complex with more advanced type system. An example would be Common Lisp's `map` function [0] (it takes a number of sequences and a function that has as many parameters as there are sequences). It would be hard to come up with a type for this in Java, and it…

Haskell's not too bad once you understand ZipList http://learnyouahaskell.com/functors-applicative-functors-an... max ZipList [1,2,3,4,5,3] ZipList [5,3,1,2] > [5,3,3,4]

I will start by saying it took me a while to even parse the expression you provided. Whoever thought that inventing new operators is a way to write readable code should really be kept far away from programming languages. The article you provided didn't even bother to give a name to and so I could at least read them out to myself.

Anyway, bitter syntax sugar aside, the way you wrote the function I proposed was... a completely different function with similar results, which does not have the type I was asking for, and you only had to introduce 2 or 3 helper functions and one helper type to do it. I wanted to work with functions and lists, but now I get to learn about applicatives and ZipLists as well... no extra complication required!

Edit to ask: could this method be applied if you didn't know the number of lists and the function at compile time? CL's map would be the equivalent of a function that produces the expression you have showed me, but it's not clear to me that you could write this function in Haskell.

Edit2: found a paper explaining that this is not possible in Haskell, and showing how the problem is solved in Typed Scheme: https://www2.ccs.neu.edu/racket/pubs/esop09-sthf.pdf

Re: Cold Showers: For when people get too hyped up about things

#227

Earlier quoted context omitted.

I think the general feeling is that there are some code patterns that are safe and easy to do with dynamic typing, but impossible with simple type systems or more complex with more advanced type system. An example would be Common Lisp's `map` function [0] (it takes a number of sequences and a function that has as many parameters as there are sequences). It would be hard to come up with a type for this in Java, and it…

IMO, Go is never a good example in static vs dynamic type system discussions (I mean, for this case: parametric polymorphism has been around since the 70s...). The language developers themselves have repeatedly stated that its type system being very limited is intentional. See e.g. here: https://github.com/golang/go/issues/29649#issuecomment-45482... TBH, sometimes I wonder why they bothered with static typing at all…

Sure, I know Go is a low blow to static typing. But in this particular regard, Java or C# don't fare much better either.

This is not a question of just supporting parametric polymorphism, but of abstracting over the number of arguments of a function, which is not supported in almost any type system I know of; and then of matching the number of arguments received with the type of function you specified initially.

Re: Cold Showers: For when people get too hyped up about things

#228
post #142
post #41

Earlier quoted context omitted.

The aim is to have a spec that is much LESS complex than the code, written at a higher level, abstracting away details. If the spec is 5x more complex than the code then indeed there’s no point.

Here is a counter argument: http://www.pathsensitive.com/2018/10/book-review-philosophy-... My summary would be: The spec must cover all possible implementations so it is usually larger than the most simple one. An example from there: > The authors of SibylFS tried to write down an exact description of the `open` interface. Their annotated version of the POSIX standard is over 3000 words. Not counting basic machinery…

How much of that is open/close being a poor abstraction, or overloading a bunch of semi-related functionality vs a more generalizable consideration systems designed with verification in mind.

Re: Cold Showers: For when people get too hyped up about things

#229
post #60

Earlier quoted context omitted.

I think the general feeling is that there are some code patterns that are safe and easy to do with dynamic typing, but impossible with simple type systems or more complex with more advanced type system. An example would be Common Lisp's `map` function [0] (it takes a number of sequences and a function that has as many parameters as there are sequences). It would be hard to come up with a type for this in Java, and it…

In my opinion, with few exceptions, the kind of programs advocates of dynamic typing want to write that static typing would have trouble dealing with, are artificial and not the common case. (Not "map" though, I need to review that case, but "map" is definitely a common and useful function!) > Another example of many people's experience with static typing is the Go style of language Remember that a lot of backlash ag…

> In my opinion, with few exceptions, the kind of programs advocates of dynamic typing want to write that static typing would have trouble dealing with, are artificial and not the common case. (Not "map" though, I need to review that case, but "map" is definitely a common and useful function!)

I mostly agree, don't get me wrong. And it's important to note that Common Lisp's `map` functions do more than what people traditionally associate with `map` - they basically do `map(foo, zip(zip(list1, list2), list3)...)`.

Still, this is a pretty useful property, and it is very natural and safe to use or implement, while being impossible to give a type to in most languages.

C++ can do it with the template system, as can Rust with macros (so, using dynamic typing at compile time).

Haskell can make it look pretty decent (if you can stand operator soup) by relying on auto-currying and inline operators and a few helper functions. I would also note that the Haskell creators also though that this functionality is useful, so they implemented some of the required boilerplate in the standard lib already.

In most languages, you can implement it with lambdas and zips (or reflection, of course).

So I think that this is a nice example of a function that is not invented out of thin air, is useful, is perfectly safe and static in principle, but nevertheless is impossible to write "directly" in most statically typed languages.

Just to show the full comparison, here is how using this would look in CL, Haskell and C#:

    CL 
        (map 'list #'max3 '(1 3 5) '(-1 4 0) '(6 1 8)) 
    Haskell
        max3  ZipList [1 3 5]  ZipList [-1,4,0]  ZipList [6,1,8]
        OR
        () (() (() max3 (ZipList [1,2])) (ZipList [-1,4])) (ZipList [3,1])
    C#
        new int[]{1,3,5}.Zip>(new int[]{-1,4,0}, (a,b) => (c) => max3(a, b, c)).Zip(new int[]{6, 1, 8}, (foo,c) => foo(c))
Note only the CL version, out of all these languages, can work for a function known at runtime instead of compile-time. None of the static type systems in common use can specify the type of this function, as they can't abstract over function arity.

Here's a paper showing how this was handled in Typed Scheme: https://www2.ccs.neu.edu/racket/pubs/esop09-sthf.pdf

Re: Cold Showers: For when people get too hyped up about things

#230
post #148
post #68

Earlier quoted context omitted.

Haskell's not too bad once you understand ZipList http://learnyouahaskell.com/functors-applicative-functors-an... max ZipList [1,2,3,4,5,3] ZipList [5,3,1,2] > [5,3,3,4]

Yeah it's not at all complicated in Haskell. I'm not sure what GP is talking about.

I replied to the parent as well, but not only is the solution the parent showed significantly more complex than the CL version, I'm not even sure it actually does what I asked.

More explicitly, the expression there seems to rely on knowing the arity of the function and the number of lists at compile time. Basically, I was asking for a function cl_map such that:

    cl_map foo [xs:[ys:[zs:...]]] = foo  xs  ys  zs  ...
Edit: found a paper explaining that this is not possible in Haskell, and showing how the problem is solved in Typed Scheme: https://www2.ccs.neu.edu/racket/pubs/esop09-sthf.pdf
Post reply on HN