Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The eigenvector of “Why we moved from language X to language Y”

erikbern.com

Post reply on HN
Print view View original

161–170 of 193 posts

Re: The eigenvector of “Why we moved from language X to language Y”

#161

Post by lmm »

peterwwillis wrote: post #158
lmm wrote: post #59

Earlier quoted context omitted.

Then why do we see a major internet security bug that would be simply impossible in any other language every couple of months?

https://www.cvedetails.com/vulnerability-list/year-2016/mont... Note the types of vulnerabilities and the languages they use. DoS, File Inclusion, XSS, Exec Code, Dir Traversal, Priv Escalation, SQLI, Bypass, CSRF, Info leak, etc. Many of them (including ones in C) have nothing to do with memory protection. Those that do (null pointer deref, use-after-free, memory leak, buffer overflow, etc) are all trivially protect…

> consider that at least with C there are basic steps that prevent many of these from becoming problems, whereas with other languages you need a hodge-podge of different, more complicated countermeasures. C is actually easier to secure because its bugs are common and not difficult to catch by the kernel.

I think this is completely wrong. In other languages you just don't have these problems in the first place because you are memory-safe by default. It's not like there's some point-buy system where all languages have to have the same number of opportunities for bugs and not having stupid memory safety bugs means you have more complex subtle bugs instead. You just eliminate a huge proportion of your bugs.

Re: The eigenvector of “Why we moved from language X to language Y”

#162

Post by cestith »

voidlogic wrote: post #76
pjmlp wrote: post #11

Earlier quoted context omitted.

Sadly getting rid of C means getting rid of UNIX, as they are symbiotic and UNIX vendors will surely never rewrite them in anything else or replace POSiX standard.

Maybe, I think it would be possible to take a BSD or Linux (tech not a UNIX) and port the kernel space to Rust and user space to Go? Not all UNIXs are closed source.

Or maybe Minix3, XNU, L4, or Mach?

http://www.minix3.org/

https://en.wikipedia.org/wiki/XNU

https://en.wikipedia.org/wiki/Mach_(kernel)

https://en.wikipedia.org/wiki/L4_microkernel_family

Re: The eigenvector of “Why we moved from language X to language Y”

#163

Post by cestith »

jrobn wrote: post #85
voidlogic wrote: post #76

Earlier quoted context omitted.

Maybe, I think it would be possible to take a BSD or Linux (tech not a UNIX) and port the kernel space to Rust and user space to Go? Not all UNIXs are closed source.

Might as well write it from scratch without POSIX and all the lessons we've learned since then. I think rust opens the door to designing kernels that are small and tight with most of the OS stuff that was traditionally in kernel space moved into user space.

Opens which door? Minix3 is microkernel based and userland compatible with NetBSD. XNU is originally based on the Mach microkernel and has part of FreeBSD bolted on. Both are open source, as are Mach proper and L4.

Perhaps someone could start with Minix3 or Darwin rather than Linux or BSD or from scratch. Replace one component at a time in Rust, or D, or Ada...

Re: The eigenvector of “Why we moved from language X to language Y”

#164

Post by peterwwillis »

lmm wrote: post #95
JustSomeNobody wrote: post #90

Earlier quoted context omitted.

Because the vast amount of C code running the internet. That's all. Rewrite everything in and you'd see just as many security vulnerabilities.

I very much doubt that. Buffer overflows simply wouldn't happen in most languages, and those are the majority of the vulnerabilities we see.

That appears not to be the case.

  pwillis@windows:~/Downloads$ wget -q -O allitems.csv  https://cve.mitre.org/data/downloads/allitems.csv
  pwillis@windows:~/Downloads$ ( for year in `seq 1999 2016` ; do TOTAL=`cat allitems.csv | grep -v RESERVED | grep "^CVE-$year" | wc -l`; BUFF=`cat allitems.csv | grep -v RESERVED | grep "^CVE-$year" | grep -i -e "buffer.*overflow\|overflow.*buffer" | wc -l`; PERCENT=`awk "BEGIN{print $BUFF/$TOTAL*100}" | cut -d. -f1`; echo "Year $year: $TOTAL CVEs, $BUFF buffer overflow related, $PERCENT% total" ; done )
  Year 1999: 1573 CVEs, 307 buffer overflow related, 19% total
  Year 2000: 1237 CVEs, 250 buffer overflow related, 20% total
  Year 2001: 1540 CVEs, 278 buffer overflow related, 18% total
  Year 2002: 2370 CVEs, 481 buffer overflow related, 20% total
  Year 2003: 1519 CVEs, 346 buffer overflow related, 22% total
  Year 2004: 2670 CVEs, 470 buffer overflow related, 17% total
  Year 2005: 4686 CVEs, 519 buffer overflow related, 11% total
  Year 2006: 7047 CVEs, 612 buffer overflow related, 8% total
  Year 2007: 6510 CVEs, 868 buffer overflow related, 13% total
  Year 2008: 7034 CVEs, 619 buffer overflow related, 8% total
  Year 2009: 4888 CVEs, 589 buffer overflow related, 12% total
  Year 2010: 4954 CVEs, 419 buffer overflow related, 8% total
  Year 2011: 4441 CVEs, 428 buffer overflow related, 9% total
  Year 2012: 5219 CVEs, 399 buffer overflow related, 7% total
  Year 2013: 5731 CVEs, 382 buffer overflow related, 6% total
  Year 2014: 7494 CVEs, 326 buffer overflow related, 4% total
  Year 2015: 6526 CVEs, 357 buffer overflow related, 5% total
  Year 2016: 7180 CVEs, 471 buffer overflow related, 6% total
According to this really shitty review of CVEs, buffer overflow is less than 10% (recently less than 6%) of tracked vulnerabilities. That's still a lot, of course.

Re: The eigenvector of “Why we moved from language X to language Y”

#165

Post by kilon »

I always thought that there is no such thing as best programming language in the world. After all they only recycle the same recipe , again and again. Then I found out Smalltalk and realized how wrong I was.

There is nothing that comes close that can compete with the massive success that Smalltalk has been. Its blows my mind how it can be so much better than anything else out there including the usual suspects (Lisp, haskell, blah blah).

But in the end its not about the language , its about the libraries. Hence why Python remains my No1 choice.

In the end however even Smalltalk is terrible outdated. The state of software is in abysmal condition trapped in its own futile efforts of maintaining backward compatibility, KISS and do not reinvent the wheel.

In sort software is doing its best to keep innovation at a minimum and as such pretty much everything sucks big time and is still stuck in stone age.

I once considered becoming a professional coder working in a company doing the usual thing, I am glad I was wise enough not to choose that path. I would have killed myself right now with all this nonsense that makes zero logical sense.

But my hope is in AI, the sooner we get rid of coders, the better. Fingers crossed that is sooner than later. Bring on our robotic overlords.

Saying that I know a lot of people that really love coding and respect it as an art and science, so there is definitely hope.

Re: The eigenvector of “Why we moved from language X to language Y”

#166

Post by erikbern »

omaranto wrote: post #144
erikbern wrote: post #125

Earlier quoted context omitted.

Author here. You are absolutely right. As I mentioned in the notes, I think this matters a bit less than it might seem like (the stationary distribution does not change if you add a diagonal matrix) but clearly some languages will have a higher propensity for people to stay. I think this flaw is even smaller than the issue of using Google statistics to infer transition probabilities. It's just a shitty proxy, at best…

That bit about the stationary distribution not changing if you add a diagonal matrix sounds completely wrong to me. Let me see if I understand what you mean. Given a matrix M with non-negative entries (and no row of just zeros), let S(M) denote the stochastic matrix you get by normalizing each row of M. You are saying that if M is any matrix and D is a diagonal matrix with non-negative entries then S(M) and S(M+D) ha…

Right. Both the matrix S and the identity matrix will project the stationary distribution onto itself. So any linear combination of them will project the stationary distribution onto itself. Let me know if I'm saying something really stupid

Re: The eigenvector of “Why we moved from language X to language Y”

#167

Post by DocSavage »

thaumasiotes wrote: post #153
j2kun wrote: post #135

Earlier quoted context omitted.

The thing I like most about this post is that it's falsifiable. We will know in ten years whether C and Java are still popular, and whether Go succeeds in the sense this data suggests. So thank you for being concrete and clear, even if it's all in fun and other people don't like it :)

> whether Go succeeds in the sense this data suggests An interesting thing about this methodology is that it is extremely sensitive to the age of a language. It's possible to switch from an old language to a new language, but not the other way around -- so if you happen to do your measurements after a language has had some uptake but before it's been around for long enough that people have built significant projects…

The sorted stochastic matrix shows that C contradicts your assumption that it's not possible to switch from a new language to an old one. Or, at least, it shows that portions of new language code are occasionally rewritten in C.

Re: The eigenvector of “Why we moved from language X to language Y”

#168

Post by omaranto »

erikbern wrote: post #166
omaranto wrote: post #144

Earlier quoted context omitted.

That bit about the stationary distribution not changing if you add a diagonal matrix sounds completely wrong to me. Let me see if I understand what you mean. Given a matrix M with non-negative entries (and no row of just zeros), let S(M) denote the stochastic matrix you get by normalizing each row of M. You are saying that if M is any matrix and D is a diagonal matrix with non-negative entries then S(M) and S(M+D) ha…

Right. Both the matrix S and the identity matrix will project the stationary distribution onto itself. So any linear combination of them will project the stationary distribution onto itself. Let me know if I'm saying something really stupid

Oh, you meant "multiple of the identity matrix" when you said "diagonal matrix"?

Re: The eigenvector of “Why we moved from language X to language Y”

#169

Post by mi100hael »

thehardsphere wrote: post #118
fishnchips wrote: post #100

Am I reading this incorrectly, or there is more movement from Swift to Objective-C than the other way around? Do I sense a methodological error?

I had this exact reaction to the graphs that also showed: 1. Movement from Postgres to MySQL 2. Movement from Mariadb to MySQL (and NOT the other way around?!?) 3. Movement from PHP to Java (I remember the sort of people leaving Java for PHP 10 years ago, and I don't think they'd go back, or that PHP people would pick Java as their choice to move to) I think maybe he has the axes labeled wrong?

MySQL is actually seeing a resurgence as people realize that ACID is valuable and performance is just fine for 99.9% of use-cases.

And Java is seeing a bit of a resurgence as well as people get fed up with shitty PHP and other dynamically typed languages. Java has some frameworks like Dropwizard and Spring Boot that make it not as terrible anymore.

Re: The eigenvector of “Why we moved from language X to language Y”

#170

Post by psiclops »

pmarreck wrote: post #140
erikbern wrote: post #125

Earlier quoted context omitted.

Author here. You are absolutely right. As I mentioned in the notes, I think this matters a bit less than it might seem like (the stationary distribution does not change if you add a diagonal matrix) but clearly some languages will have a higher propensity for people to stay. I think this flaw is even smaller than the issue of using Google statistics to infer transition probabilities. It's just a shitty proxy, at best…

And I share my Go reservations with you, given the whole error-handling (or lack thereof) philosophy as well as information emerging that it may require 100 lines of Go to do roughly the same amount of work as 20 lines of Elixir or Haskell, according to one example at https://medium.com/unbabel-dev/a-tale-of-three-kings-e0be17a... (Although I concluded the Haskell-Elixir equivalency myself based on functional semanti…

Go is verbose. There is a lot of thought behind that, but that is an intentional design aspect of the language. Personally I would not use gin, either net/http, gorilla/mux, or httprouter are solid choices
Post reply on HN