Things I Regret About Node.js [video] • Hacker News Board

Re: Things I Regret About Node.js [video]

#401

Post by zbjornson » Thu, Jun 07, 2018, 1:26 PM UTC

mschuetz wrote: post #345

blattimwind wrote: post #123

Earlier quoted context omitted.

> (File reads were cached by OS and/or disk due to repeated testing, but file writes probably not) Unless you flush the pages manually, your dirty pages (written files) live on long after your process died. Depending on system and configuration, minutes or even hours can pass before they are flushed to disk.

Do you know about best practices to benchmark with uncached data? It's something I've been wondering about for a long time and I've seen many attempts at benchmarking things without regard for disk caching. e.g. benchmarking in-memory data bases to out-of-core databases but because of caching due to repeated runs, the results were meritless since the out-of-core databases had their stuff in memory as well.

Take a look at https://github.com/Feh/nocache. It works effectively for reads. I don't know about writes.

Re: Things I Regret About Node.js [video]

#402

Post by zbjornson » Thu, Jun 07, 2018, 1:32 PM UTC

mschuetz wrote: post #153

brlewis wrote: post #137

Earlier quoted context omitted.

Did you use typed arrays, or did the benefit of async I/O outweigh the cost of using double-precision floats for everything?

I'm reading the values directly from the Buffer object that is returned from fs.readFile using buffer.readUInt32LE and buffer[index], then convert the values to doubles to do the hit-test and if it succeeds, it's immediately written to the output buffer. The nodejs Buffer object is a subclass of Uint8Array as far as I know. On writing, the double coordinate values are transformed back into a fixed-precision integer f…

Just constructing a Uint32Array view of the buffer and accessing values by index way(!) faster than using the buffer.read* methods.

Likewise, you don't need to copy bytes in your last code sample. You can copy uint32s.

Re: Things I Regret About Node.js [video]

#403

Post by ppeetteerr » Thu, Jun 07, 2018, 1:36 PM UTC

pknopf wrote: post #274

jrochkind1 wrote: post #219

Earlier quoted context omitted.

I think this is a story that gets repeated lots of times in our world of open source software dev. 1. X is SO bloated and poorly engineered full of bad legacy decisions. 2. We can totally do better let's invent a new thing, Y! 3. Wow, Y is so clean and fast and understandable. 4. But it doesn't do this thing a bunch of people reasonably really need... let's add it. (repeat 3 and 4 a few hundred times) 5. Y is so bloa…

How do you break the loop?

You have strong leadership that makes good decisions.

Re: Things I Regret About Node.js [video]

#404

Post by cloverich » Thu, Jun 07, 2018, 1:46 PM UTC

davedx wrote: post #395

always_good wrote: post #204

Earlier quoted context omitted.

I'd say your intel is far out of date. Async-everything + async/await makes Node more elegant than the same programs in Ruby/Python. Even little things like "make these two database requests in parallel and wait on them both" or "process these urls but only have 8 requests in flight at a time."

> "process these urls but only have 8 requests in flight at a time." What's the state of the art solution to this in node.js?

There's many ways to do this, but at its simplest because Node is single threaded, you can have a shared array of URL's and then a pool of workers who .pop() a URL off and stop when there are none left. Each worker's http request runs async, so with the `async` keyword at the start of their signature, and `await fetch(url)` inside, the code reads (and in some ways behaves) like regular synchronous iteration but runs (mostly) concurrently.

Re: Things I Regret About Node.js [video]

#405

Post by ppeetteerr » Thu, Jun 07, 2018, 1:55 PM UTC

wolco wrote: post #221

dnomad wrote: post #144

Earlier quoted context omitted.

Watching the Javascript poorly reinvent the wheel has been very disappointing. Very simple mistakes like immutable, never changing build releases that Java developers understood 15 years ago are become recent front page news in this community. Ironically, even though all the code is open-source pre-existing knowledge does not get leveraged in the open source world. There's a kind of market failure at work here it see…

I like the energy around the javascript everywhere movement. So what if they reinvent the wheel, sometimes you find a better wheel and break the rules along the way. There is something exciting about developers using a language in ways it was never designed. Then having the language change to support the changing ecosystem...

It's when you build a business on a technology and then have to re-invest to rebuild the product, that's when it becomes an issue. Think of all the start ups that built running businesses on Angular 1.

Re: Things I Regret About Node.js [video]

#406

Post by jrochkind1 » Thu, Jun 07, 2018, 1:55 PM UTC

pknopf wrote: post #274

jrochkind1 wrote: post #219

Earlier quoted context omitted.

I think this is a story that gets repeated lots of times in our world of open source software dev. 1. X is SO bloated and poorly engineered full of bad legacy decisions. 2. We can totally do better let's invent a new thing, Y! 3. Wow, Y is so clean and fast and understandable. 4. But it doesn't do this thing a bunch of people reasonably really need... let's add it. (repeat 3 and 4 a few hundred times) 5. Y is so bloa…

How do you break the loop?

I think you gotta have a good understanding of the domain and use cases you want to hit (which is really hard, especially so when it's a general purpose programming language whose domain is... everything), and design from the start with a vision of hitting those use cases, instead of having to shoe-horn them in later.

Of course, use cases will still evolve, and your initial understanding is always flawed, there's no magic bullet, designing general purpose software (or a language or platform!) meant to hit a wide swath of use cases flexibly is _hard_.

And then, yeah, like others have said, you need skilled, experienced, and strong leadership. You need someone (or a small group of people) who can say 'no' or 'not yet' or 'not like this' to features -- but also who can accurately assess what features _do_ need to be there to make the thing effective. And architects/designers who can design the lower levels of abstraction solidly to allow expected use cases to be built on top of them cleanly.

But yeah, no magic bullet, it's just _hard_.

As developer-consumers of software, we have to realize that _maturity_ is something to be valued, and a trade-off for immature but "clean" is not always the right trade-off -- and not to assume that the immature new shiny "clean" thing will _necessarily_ evolve to do everything you want and be able to stay that way. (On the other hand, just cause something is _old_ doesn't _always_ mean it's actually _mature_ or effective. Some old things do need to be put down). But resist "grass is always greener" evaluation that focuses on what you _don't_ like about the original thing (it's the pain points that come to our attention), forgetting to take into account what it is doing for you effectively.

Re: Things I Regret About Node.js [video]

#407

Post by rubber_duck » Thu, Jun 07, 2018, 1:58 PM UTC

Quarrelsome wrote: post #372

rubber_duck wrote: post #369

Earlier quoted context omitted.

C# is very verbose and tedious compared to more expressive languages - having to deal with CLR types/API at runtime while using a language with very limited expressiveness (C#) is not very productive. It's better than Java if that's what you're aiming at - but JVM has an incredible ecosystem of stuff that works - much larger than .NET core which is not very mature in many areas (recently had to revert to .NET 4.7 bec…

> using a language with _very_ limited expressiveness (C#) is not very productive. o_0. Think you need to check yourself mate. I believe the productiveness of more "expressive" language tends to be undermined by the loss of productivity that occurs when you're compelled to write blog posts or comment on hacker news about how amazingly productive and expressive your language is.

I can do like 3-4 hours of productive work a day realistically - after that I lose focus. I can push this in some periods - but that's the ammount of time I limit myself to be functional over long term.

If I need to waste that time sifting trough boilerplate than I'm pretty upset because I get less shit done in that time window.

Chatting on forums is a casual brain teaser and keeping up to date on industry stuff.

Re: Things I Regret About Node.js [video]

#408

Post by pjmlp » Thu, Jun 07, 2018, 2:03 PM UTC

xab9 wrote: post #399

CSMastermind wrote: post #394

Earlier quoted context omitted.

npm is the worst software I use daily. * The maintainers have pushed several breaking updates by mistake (I'm a teapot recently). * There have been a few cascading failures due to the ecosystem (leftpad). * node-gyp (alluded to in the talk) break cryptically on install in different operating system/package combinations. It also obscures the actual package contents. * The lack of package signatures and things like nam…

I can't argue with you, but "most of the time" it works. Not that I have a choice (unless I stop being a frontend developer), but the time we spend with node/npm debugging is not critical in the timesheet/log.

There is a choice, I do native frontend.

Re: Things I Regret About Node.js [video]

#409

Post by overcast » Thu, Jun 07, 2018, 2:03 PM UTC

BigJono wrote: post #254

overcast wrote: post #125

Earlier quoted context omitted.

You don't see how having to know only one language, is easier/faster than having to know two languages? A lot of people are doing both front and backend work. Often times it's a single person running the show. I'll tell you right now I use node for ALL my web projects, because I only have to focus on a single language. When I wear every hat managing the domain, server, databases, mail, user questions, and everything…

I don't see it. Any old idiot can learn a programming language. The difficulty doesn't come from the language, it comes from the environment and tooling.

So what's your alternative? I don't know of any environment easier than installing Node. I type 'n latest', and yarn add/install. Done.

Ruby? Laughable. PHP has a zillion dependencies. Go with all of its system paths. Java VM, no thank you.

Re: Things I Regret About Node.js [video]

#410

Post by alirobe » Thu, Jun 07, 2018, 2:03 PM UTC

royjacobs wrote: post #170

cpeterso wrote: post #133

Earlier quoted context omitted.

That's a good example of the Innovator's Dilemma: the enterprise incumbent is unseated by some "crappy" lightweight solution that is easier to get up to speed and solves enough of the problem. The complexity, accidental and essential, comes later.

Yes, this is exactly what's happening. Existing tools are seen as too complex because people don't seem to be realizing that the complexity is not accidental, but necessary. I'm not trying to say that Java has no accidental complexity of course, I don't want to open that can of worms :)

Which complexity is actually necessary? Does it change when you have 400gbit, SSDs, and watchOSes? How about 1TB of core memory? If we aren't wrangling with handling 75 spinning disk's connected to a 10mbit network with 13" blurry CRT monitors, perhaps we don't need to discuss the finer points of engineering efficient client/server LOB applications? Perhaps we ought to discuss MDM, RF, ML, and lifestyle impacts.

This is all just the process of evolution at play. What seems obvious today wasn't yesterday - applies to biology, material science, medicine, engineering, art, music, architecture, design, taxi services, marketing, government, politics, and so why shouldn't it be so in computing?

Sidenote: I love the humility of this video. I remember the days when node was first unleashed. I could not have imagined how it has changed the way we all work. It all seemed so obvious from day one, and here we are today. What a brilliant contribution.