Generalizing 'jq' and Traversal Systems using optics and standard monads • Hacker News Board

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#41

Post by AnthonBerg » Wed, Oct 07, 2020, 9:09 PM UTC

synthc wrote: post #39

ben509 wrote: post #34

Earlier quoted context omitted.

The theory lets compilers catch a ton of bugs at compile time. When you can get a functional language to type check, it really does Just Work. And when you limit side-effects, you know another function isn't going to monkey with something it's not supposed to. That said, I'm presently struggling with AST transformations and other fun stuff in Haskell, so I completely agree with stating it imperatively. It's far, far…

>When you can get a functional language to type check, it really does Just Work. Not all functional languages have static typing. Also type checking helps, but saying that if the types check out it just work it pushing it IMO. No type checker will catch this error: sqrt :: double -> double sqrt x = x sqrt 10

Dependent types are able to express the constraints that prevent or catch this error.

The types of the arguments to the function can have value constraints on it, and those constraints can be determined from a value that exists there: the name of the function.

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#42

Post by mgraczyk » Wed, Oct 07, 2020, 9:11 PM UTC

vortico wrote: post #20

mgraczyk wrote: post #10

Earlier quoted context omitted.

That's more difficult to type than something like `.company.staff | .pets | flatten | filter .type == "cat" | log(...)`

Wait a second, you left out the most important part: What's in the `log(...)` function? How do you get the name of the pet and the name of the staff in that line if they have the same accessor `.name`? That's my point. The fact that you didn't include it implies to me that it was too difficult to figure out, and a true implementation would be far more complicated, so it's not a fair comparison.

I disagree, there are many ways to solve this problem. for example

   .company.staff | .pets | flatten | filter .type == "cat" | print(.name + " belongs to " + ..name)

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#43

Post by oillio » Wed, Oct 07, 2020, 9:16 PM UTC

vortico wrote: post #15

choeger wrote: post #11

Earlier quoted context omitted.

Functions are composable. Your statement is not. If I would ask you to create a snippet that searches for cats, another one that searches for dogs and a third one that searches for staff that owns both a cat and a dog you will either duplicate a lot of code or end up with functions. Now if you restrict yourself to not introduce any side effects, the interpreter can be proven correct, the execution can be analysed muc…

What's wrong with functions? A function `printCatsBelongingToStaff()` is much easier to read than a line of functional code. I don't understand what you mean by "analyzed much better" and "neat safety guarantees". Is my code hard to analyze or unsafe?

A functional coding technique is strictly more powerful.

Say we want to reuse your cat search code. Sometimes we want to log it, and sometimes we want to export it as a different data structure.

You could write a function that took a company and returned a list of pets.

What if you wanted to take the first 10 of 10 million. Returning the full list wouldn't be very efficient, you would need to alter your function to support limits.

What if you were then given a list of companies. You would need to modify the function, or write a wrapper.

What if it isn't a list anymore but an async stream of companies. More modifications.

With functional techniques, we can abstract the concept of "filtering a company for pets" without worrying about the concrete implementation of how the returned pets are stored. This allows the one function to be reused efficiently in many different applications, in ways not possible with imperative code.

This is just one example of abstraction that is possible. Functional techniques are not a good fit for all problems, but they are a good tool to have in your toolbox as they make some problems much easier to reason about.

Paul Graham has written a lot on the power of functional languages. OnLisp is a good place to start: http://www.paulgraham.com/onlisp.html

Or some of his early blog posts discuss these concepts: http://www.paulgraham.com/articles.html

Another classic that will change the way you look at code design is Structure and Interpretation of Computer Programs: https://web.mit.edu/alexmv/6.037/sicp.pdf

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#44

Post by formerly_proven » Wed, Oct 07, 2020, 9:48 PM UTC

whateveracct wrote: post #37

formerly_proven wrote: post #3

"Optics" and "lenses" seem like such bad names just to be able to make a "view data through a lens" pun.

They're good names for a couple reasons 1. The analogies made do hint at their meanings. 2. But at the same time, the names are more proper nouns than definitions. For something so abstract, it's better to give it an opaque, easy-to-remember name than to give it a "better" name that maybe oversimplifies what it is. This is basically the "Functor" vs "Mappable" argument.

They're practically useless for searching for them. Unsurprisingly trying to co-opt widely used terms tends to be ambiguous and confusing (cf. crypto cryptography, cryptocurrency, and those are at least related). Perhaps the only exception in this group is "data lens", which works as a compound [1]. Guess what optics software does? No ETL, that's for sure. Optics patterns? Uh-oh. Optics development? Nuh-uh. Optics data? Turns out there are multiple companies with that name, and they're not doing ETL. Optics library? Image libraries of lenses (the kind you can touch). Optics computing? Sorry, that's a synonym of optical computing. Optics design? That's the engineering of optical systems. Optics scheme? Guess what arrangements of lenses are called... Optics programming? Aha, now the topic at hand actually manages to be in the majority of the results - barely, since four out of ten are still about that other thing humans have been at for a couple hundred years. Optics ? Things for doing optical computations and simulations.

[1] Also follows the rule of "if you're going to name something after a common word, maybe give it a qualifier not usually used with that word to make it unique and easy to find". Examples: sphinx (doc, search), click (cli), pyramid (web framework), pacman (linux), language (programming).

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#45

Post by galkk » Wed, Oct 07, 2020, 10:04 PM UTC

While I think it's closer to be dead on arrival (PartiQL), I like PartiQL/GoogleSQL way (generalizing SQL over nested structures) much more than the examples in the article

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#46

Post by imoverclocked » Wed, Oct 07, 2020, 10:08 PM UTC

synthc wrote: post #36

choeger wrote: post #22

Earlier quoted context omitted.

Your code is impossible to analyze. Console.log is a side effect. For loops can be easily mapped into functional constructs, but not your snipped does not compute anything. It's not a function.

Don't be pedantic, this is super simple code that is easy to analyze. Side-effects are not the terror that haskell programmers make of it.

After working on and maintaining several very large codebases, I would strongly disagree. When large blocks of code start being called only for their side-effects then refactoring/deleting old code gets pretty tricky.

Conversely, when writing green-fields code, side effects are extremely convenient and make developers more productive in the very short-term.

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#47

Post by rrdharan » Wed, Oct 07, 2020, 10:17 PM UTC

vortico wrote: post #13

mgraczyk wrote: post #10

Earlier quoted context omitted.

That's more difficult to type than something like `.company.staff | .pets | flatten | filter .type == "cat" | log(...)`

Maybe so, but much easier to read . Code is read 10x more often than written, especially in teams, so it makes sense to optimize for legibility rather than elegance in some "higher" mindset, or less keys pressed (because most of the time will be spent staring at the line you wrote than actually typing if you write a functional line like that.)

Forget it, Jake. It’s Chinatown.

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#48

Post by emteycz » Wed, Oct 07, 2020, 10:29 PM UTC

warkdarrior wrote: post #38

Daishiman wrote: post #8

Earlier quoted context omitted.

LINQ is definitely a precursor, and has tons of fantastic ideas. It's unfortunate that Microsoft didn't continue to invest heavily into this as it's rarely mentioned nowadays.

Wikipedia claims there are LINQ implementations for Javascript/Typescript and for PHP (in addition to C#, of course).

These are really just simplified versions, though.

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#49

Post by specialist » Wed, Oct 07, 2020, 10:37 PM UTC

vortico wrote: post #9

Why do you need all this theory when you can use simple imperative languages? for staff in company.staff: for pet in staff.pets: if pet.type == 'cat': print(pet.name + " belongs to " + staff.name) I don't understand why functional languages are used at all.

I like both imperative and functional style programming. They're isomorphic if composition is favored. Imagine interpreters iterating over ASTs. Easy peasy.

I strongly dislike hybrids, multiparadigm, metaprogramming, and inheritance-heavy OOAD.

Most of our work is CRUD, meaning data centric. Right?

My arm wavy overgeneratalization is that functional programming promotes functional decomposition style programming, which emphasizes flow of control modeling over data modeling. Especially when noobs go overboard with lambdas, stream processing, reducers, and so forth, which obfuscates the data transformations.

Whereas imperative programming can be more straightforward for simple data processing. Until noobs go overboard with OOAD stuff like inheritance, making wrapper classes for everything, and design pattern overkill. Then the wheels fall off and the engine explodes once someone reads a book about inversion of control, dependency injection, mocking, aspects, or optionals (non-null).

YMMV.

Re: Generalizing 'jq' and Traversal Systems using optics and standard monads

#50

Post by matt-noonan » Wed, Oct 07, 2020, 10:53 PM UTC

vortico wrote: post #9

Why do you need all this theory when you can use simple imperative languages? for staff in company.staff: for pet in staff.pets: if pet.type == 'cat': print(pet.name + " belongs to " + staff.name) I don't understand why functional languages are used at all.

The point is really that lenses are values that represent locations in a data structure. And, as values, they can be combined, transformed, serialized, etc etc. Imagine having a type that represents a chain of method selectors, and that gives you some idea of the purpose.

The fact that method selectors only appear very rarely as first-class values in most languages means that most people aren’t tuned in to scenarios where they could be applied. But I bet you’ve invented special cases of this yourself, when you had a function that needed to dig data out of one of several locations, depending on other inputs.