Why programming languages matter [video] • Hacker News Board

Re: Why programming languages matter [video]

#41

Post by PartiallyTyped » Sat, Oct 28, 2023, 6:34 AM UTC

justinpombrio wrote: post #20

mjfl wrote: post #4

Very little innovation in programming languages has happened regarding new realities at the hardware level especially transition from serial to parallel execution.

Some relevant PL research STM: https://cs.brown.edu/~mph/HerlihyM93/herlihy93transactional.... Region based memory management: https://en.wikipedia.org/wiki/Region-based_memory_management Rust lifetimes, originally from Cyclone: https://en.wikipedia.org/wiki/Cyclone_%28programming_languag... Mutable value semantics: https://arxiv.org/pdf/2106.12678.pdf

Rust compiler enables trivial parallelism by enforcing multiple readers ^ writer; and it’s beautiful.

See Rayon.

Re: Why programming languages matter [video]

#42

Post by mrkeen » Sat, Oct 28, 2023, 6:43 AM UTC

nomel wrote: post #6

mjfl wrote: post #4

Very little innovation in programming languages has happened regarding new realities at the hardware level especially transition from serial to parallel execution.

Does anyone have any good counterpoints to this? From my, naive, perspective, this seems to be relatively true. I've always assumed that, by now, I would be able to write code, in a semi mainstream language, and it would be made somewhat parallel, by the compiler. No need for threads, or me thinking of it. There's projects like https://polly.llvm.org , but I guess I assumed there would be more progress through the de…

One of the prerequisites to this is for the mainstream to stop doing things which are incompatible with concurrency.

Mutation (and other effects) makes the order of computations important. If you're writing to and reading from variables, the compiler is not free to move those operations around, or schedule them simultaneously.

And you probably don't want to be rid of all mutation. So what if you separated the mutating from the non-mutating? Well you'd need a sufficiently powerful type system. Likely one without nulls - as they can punch a hole through any type checking.

If you want this stuff in the mainstream, you at least have to get all the nulls and mutation out of the mainstream, which I don't think will happen.

The industry for the most part heeded "goto considered harmful" (1968), but hasn't done so with "the null reference is my billion dollar mistake (2009)". Maybe we just have to wait.

Re: Why programming languages matter [video]

#43

Post by recursivecaveat » Sat, Oct 28, 2023, 6:51 AM UTC

krick wrote: post #40

synergy20 wrote: post #13

Earlier quoted context omitted.

same here, some companies are hiring a compiler team for AI in fact. I was told Rice and UIUC provide the best compiler program, though not necessarily AI related, should be similar though.

What does it mean, specifically, "compiler team for AI"? I get it that it's some hot new trend from the last 2 posts, but I'm struggling to imagine what exactly the perfect product should look like and why everyone wants it so bad, allegedly.

It's just about compiling a neural net down to run as efficiently as possible. Either on GPU, CPU, or your own accelerator. Neural nets are very computational intensive, while being pretty uniform internally and as a class. Everyone making silicon and a fair few other companies as well has an AI compiler team right now. At the moment the hot product would just be LLM tokens for as few cents each as possible.

Re: Why programming languages matter [video]

#44

Post by bmacho » Sat, Oct 28, 2023, 6:55 AM UTC

eslaught wrote: post #16

nomel wrote: post #6

Earlier quoted context omitted.

Does anyone have any good counterpoints to this? From my, naive, perspective, this seems to be relatively true. I've always assumed that, by now, I would be able to write code, in a semi mainstream language, and it would be made somewhat parallel, by the compiler. No need for threads, or me thinking of it. There's projects like https://polly.llvm.org , but I guess I assumed there would be more progress through the de…

It's not totally clear what you're looking for. As you noted, polyhedral compilers work on a pretty restricted subset of programs, but are fairly impressive in what they do. There has been research on distributed code generation [1] as well as GPUs [2]. While there has been work on generalizing the model [3], I think the amount of parallelization that a compiler can do is still very limited by its ability to analyze…

There is also HVM [9], which can run sequentially written code parallel for some degrees. (It can run parallel some sequential Haskell code naively transpiled to HVM that GHC doesn't parallelize.)

[9] : https://github.com/HigherOrderCO/HVM

Re: Why programming languages matter [video]

#45

Post by jokoon » Sat, Oct 28, 2023, 11:32 AM UTC

krick wrote: post #40

synergy20 wrote: post #13

Earlier quoted context omitted.

same here, some companies are hiring a compiler team for AI in fact. I was told Rice and UIUC provide the best compiler program, though not necessarily AI related, should be similar though.

What does it mean, specifically, "compiler team for AI"? I get it that it's some hot new trend from the last 2 posts, but I'm struggling to imagine what exactly the perfect product should look like and why everyone wants it so bad, allegedly.

I believe it's either related to the new AI-specialized chips, or maybe to the factoring or neural network graphs.

Any specialized domain tend to have its own domain specific language, so obviously it would be true for AI, too.

Re: Why programming languages matter [video]

#46

Post by epgui » Sat, Oct 28, 2023, 1:59 PM UTC

sidlls wrote: post #39

epgui wrote: post #28

Earlier quoted context omitted.

> all problems will be made to look like math problems All problems that can be solved with code are math problems. Proofs and programs are isomorphic (see the Curry-Howard correspondence). --- Edit: this is a factually accurate comment, delivered dispassionately. It's not controversial or new-- it's something we've known for longer than the C language has existed. Why the downvote? Like I said, see this: https://en.…

This is like saying all physical engineering problems are quantum mechanics (or insert other physics theory) problems. It’s technically correct (the best kind of correct), but misleading and useless as a matter of practice.

I see what you’re trying to do with the comparison, but it’s not really the same.

In your example, the two things are separated by at a minimum one layer of emergence: your example is more like saying biology is just chemistry. In maths and programming, they are both at the same level, no emergence.

I also haven’t found what you say to be true at all— As I’ve been learning more maths and more programming, and learning more about the link between the two, I have found that the ability to see problems from more than one angle has had a dramatic impact on how clearly I think and how efficiently I solve problems. Not useless whatsoever.

Re: Why programming languages matter [video]

#47

Post by arzke » Sat, Oct 28, 2023, 2:05 PM UTC

epgui wrote: post #34

roenxi wrote: post #31

Earlier quoted context omitted.

I find this whole thread fascinating because nobody seems to have identified what functional programming is defined as (programmers can just use functions to code and end up with functional programming, it isn't an obscure style), who the advocates are or what they claim. And yet, without any substance, the debate rages.

> nobody seems to have identified what functional programming is defined as Nobody asked (until now, so thank you for asking!), and this is a fairly well discussed topic for anyone who cares to search! You will probably get a lot of slightly different answers depending on who you ask or where you look, but I think a very strong common thread is "referential transparency". Functional programming gives you that, and th…

> Referential transparency is related to the concept of "function purity" (in the sense that either one usually guarantees the other), which you will often hear people talk about as well.

Isn't referential transparency (the property of a function that allows it to be replaced by its equivalent output) a consequence of function purity? In other words: could a pure function not be referentially transparent?

Also, I remember Robert C. Martin describing the functional programming paradigm as a "restriction upon assignment". I kind of like this definition as the rest seems to flow from it: if you can't assign, you can't mutate. If you can't mutate, you have pure functions, which are referentially transparent.

Re: Why programming languages matter [video]

#48

Post by pdimitar » Sat, Oct 28, 2023, 2:09 PM UTC

Programming languages do matter but not as much as many people think f.ex. the HN crowd has a soft spot for LISP, and most of the don't even have proper parallel execution (and no I am not talking OS threads, having direct access to those should honestly be removed at one point). And sure, Racket and a few others are "Working on having an actor model". Wake me up when they achieve it. I am betting on somewhere in the 2030s, best case scenario.

A combination of a good and restrictive language compiler (Rust, OCaml, Haskell) and an amazing runtime (Erlang) is the sweet spot that everyone should be aiming at.

If anything, I am very tired of seeing yet another LISP dialect -- or any other language really, come to think of it now -- being announced. Many of us the programmers love to play and going to check on these languages is IMO taking away precious mind-share.

If anything, in my eyes it's exactly because programming languages matter is the reason why we should have less of them. We should start folding some languages inside others. (Or abandon them.)

Re: Why programming languages matter [video]

#49

Post by AnimalMuppet » Sat, Oct 28, 2023, 2:13 PM UTC

epgui wrote: post #28

sidlls wrote: post #19

Earlier quoted context omitted.

"FP" camps tend to come in two flavors: "I'm a mathematician writing a computer program, and all problems will be made to look like math problems even if it means the program becomes an inscrutable mess of types and dense syntax" and "functional-ish idioms are included". The latter is useful, sometimes, for cloud computing and parallel computation; the former tends to have too many problems (slow build, slow executio…

> all problems will be made to look like math problems All problems that can be solved with code are math problems. Proofs and programs are isomorphic (see the Curry-Howard correspondence). --- Edit: this is a factually accurate comment, delivered dispassionately. It's not controversial or new-- it's something we've known for longer than the C language has existed. Why the downvote? Like I said, see this: https://en.…

Factually accurate (to some degree), but almost completely irrelevant.

First, not all problems that can be solved with code are math problems. Take driving a serial port, for instance. There might be some aspects of it that are mathematical, but mostly it's a matter of interfacing with the chip spec.

Second, even for problems that are isomorphic to a math problem... the thing about isomorphisms is that they aren't identical. One form or the other is often easier to deal with. (That's one of the reasons we care about isomorphisms - we can turn hard problems into easier ones.)

Well, which is easier, writing programs or doing proofs? Almost always, writing programs is easier. This is why it's (almost) completely irrelevant.

Nobody wants to write code that way. So nobody cares. They're not going to care, either, no matter how forcefully you point out Curry-Howard.

Now, as you say elsewhere, you can gain insights from math that can change your code. That's true. But I suspect that most of the time, you don't actually write the code as a proof.

Re: Why programming languages matter [video]

#50

Post by AnimalMuppet » Sat, Oct 28, 2023, 2:17 PM UTC

charcircuit wrote: post #18

epgui wrote: post #17

Earlier quoted context omitted.

Hardware was for a very long time a limiting factor in the practicality of FP. For most general applications, today this is a relative non-issue. FP is also particularly well suited for cloud computing and parallel computation.

The parent comment was talking about aligning programming languages with the hardware. I am not commenting about the viablity of those languages, but rather that if your goal is to write the most performant code by understanding the strengths and weaknesses of the hardware than using fp concepts is not the way to do it.

Well... if code were pure (in the FP sense), then a "sufficiently smart compiler" could move it around to extract the maximum performance.

But, as always, the sufficiently smart compiler never shows up. So we're left with the humans doing the tuning, and as you say, FP is kind of antithetical to that approach.