Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye, Node.js Buffer

sindresorhus.com

Post reply on HN
Print view View original

41–50 of 103 posts

Re: Goodbye, Node.js Buffer

#41

Post by mofle »

loudgas wrote: post #39

>Buffer also comes with additional caveats. For instance, Buffer#slice() creates a mutable segment linked to the original Buffer, while Uint8Array#slice() creates an immutable copy, resulting in possible unpredictable behavior. The problem is not the behavior of the Buffer#slice() method, but the fact that Buffer is a subclass of Uint8Array, but changes the behavior of an inherited method. But this is an implementati…

Completely different behavior is not an implementation detail.

Changing method behavior in subclasses is part of inheritance, but it shouldn't confuse or mislead. In the case of Buffer and Uint8Array, the altered `.slice()` functionality isn't a mere implementation detail; it's a significant deviation. This inconsistency can lead to unexpected bugs, especially for those who assume similar behavior based on the inheritance hierarchy. It's crucial for reliability that such fundamental behaviors remain predictable across subclasses.

Re: Goodbye, Node.js Buffer

#42

Post by jokethrowaway »

wayvey wrote: post #3

It's good to see that people care about native JS standards. I'm a bit concerned how Bun seems to be adding their own proprietary APIs to their JS runtime, and can't help feeling that it could be intentional vendor lock-in. Since they are venture backed I'm sure they want some ROI from the project eventually and if they start charging money it will be harder to switch away due to the proprietary APIs without refactor…

It's still OSS, not proprietary APIs. Feel free to fork it in the future, once they need to make their money back.

If it's faster than node and the API to do what I need is better (Bun.FileSystemRouter comes to mind), I don't see why not to switch over.

Heck, I would accept even less retro-compatibility to get rid of transpiling, that completely oscene political mess they did with modules and treating TS as a 2nd class citizen.

The amount of time I'm still hit by ESM vs CJS is insane. Truly a Python 2vs3 moment for the node.js community.

Re: Goodbye, Node.js Buffer

#43

Post by ravenstine »

mofle wrote: post #20
ravenstine wrote: post #11

I agree with using Uint8Array (although it's really ArrayBuffer that's the key difference), but what is up with the author using an NPM package to do something as trivial as checking if an object is a Uint8Array? `Uint8Array.prototype.isPrototypeOf` should be perfectly adequate and doesn't involve adding an attack vector to your application.

`Uint8Array.prototype.isPrototypeOf` and `instanceof Uint8Array` do not work across realms (frames, Node.js VM, etc). Feel free to copy-paste the function to your own code base if you don't want the dependency: ``` const objectToString = Object.prototype.toString; export function isUint8Array(value) { return value && objectToString.call(value) === '[object Uint8Array]'; } ```

I understand what you're saying, but that's actually in support of my point. This is still extremely trivial code to implement and, from what I can tell, doesn't warrant downloading an NPM package. Have we already forgotten the left-pad fiasco?

This isn't meant as a personal attack on anyone, but we really need to frown upon needless dependencies, especially given the growing number of malicious NPM packages.

Re: Goodbye, Node.js Buffer

#44

Post by paddy_m »

Somewhat related, who has played with performant JS dataframe libraries? I mostly want to serialize from pandas/polars to JS in a fast way, with some very minor selection and indexing operations JS side. Ideally I could find a library that does dataframe like operations from reading base64 encoded Buffers/Arrays into their JS objects, then doing the right things.

I'm investigating https://arrow.apache.org/docs/js/ https://github.com/vega/falcon https://github.com/pola-rs/nodejs-polars https://github.com/uwdata/arquero

Re: Goodbye, Node.js Buffer

#45

Post by paddy_m »

paddy_m wrote: post #44

Somewhat related, who has played with performant JS dataframe libraries? I mostly want to serialize from pandas/polars to JS in a fast way, with some very minor selection and indexing operations JS side. Ideally I could find a library that does dataframe like operations from reading base64 encoded Buffers/Arrays into their JS objects, then doing the right things. I'm investigating https://arrow.apache.org/docs/js/ ht…

I added a github issue to my project where this matters literally this morning. I'd love to talk to someone dealing with the same stuff. I have baked dataframe -> JS serialization utils so many times over the past decade. I never want to do much in JS, just simple selection and filtering. But you end up needing both sides of the serialization to do it right.

Re: Goodbye, Node.js Buffer

#46

Post by mofle »

ravenstine wrote: post #43
mofle wrote: post #20

Earlier quoted context omitted.

`Uint8Array.prototype.isPrototypeOf` and `instanceof Uint8Array` do not work across realms (frames, Node.js VM, etc). Feel free to copy-paste the function to your own code base if you don't want the dependency: ``` const objectToString = Object.prototype.toString; export function isUint8Array(value) { return value && objectToString.call(value) === '[object Uint8Array]'; } ```

I understand what you're saying, but that's actually in support of my point. This is still extremely trivial code to implement and, from what I can tell, doesn't warrant downloading an NPM package. Have we already forgotten the left-pad fiasco? This isn't meant as a personal attack on anyone, but we really need to frown upon needless dependencies, especially given the growing number of malicious NPM packages.

No one is forcing you to use it. You can choose to reimplement the code yourself or you can choose to copy-paste the code. I made the package for my own convenience as I need to transition a lot of packages from `Buffer` and I don't want to maintain duplicates of the code in every package. Others are free to use the package or not.

Re: Goodbye, Node.js Buffer

#47

Post by brundolf »

wayvey wrote: post #3

It's good to see that people care about native JS standards. I'm a bit concerned how Bun seems to be adding their own proprietary APIs to their JS runtime, and can't help feeling that it could be intentional vendor lock-in. Since they are venture backed I'm sure they want some ROI from the project eventually and if they start charging money it will be harder to switch away due to the proprietary APIs without refactor…

The accusation of malice goes a bit far, but I agree it's a problem whether it's intentional or not. Bun takes a slapdash approach to shipping features and APIs, which means things get out fast but also rubs me the wrong way re: long-term implications and vision

Re: Goodbye, Node.js Buffer

#48

Post by loudgas »

mofle wrote: post #41
loudgas wrote: post #39

>Buffer also comes with additional caveats. For instance, Buffer#slice() creates a mutable segment linked to the original Buffer, while Uint8Array#slice() creates an immutable copy, resulting in possible unpredictable behavior. The problem is not the behavior of the Buffer#slice() method, but the fact that Buffer is a subclass of Uint8Array, but changes the behavior of an inherited method. But this is an implementati…

Completely different behavior is not an implementation detail. Changing method behavior in subclasses is part of inheritance, but it shouldn't confuse or mislead. In the case of Buffer and Uint8Array, the altered `.slice()` functionality isn't a mere implementation detail; it's a significant deviation. This inconsistency can lead to unexpected bugs, especially for those who assume similar behavior based on the inheri…

Any behavior that is not defined in the spec[0] is, by definition, an implementation detail. Relying on undefined behavior is a recipe for bugs. If you need an immutable array, and the spec doesn't require the returned array to be immutable, you should create one yourself.

[0] https://tc39.es/ecma262/multipage/indexed-collections.html#s...

Re: Goodbye, Node.js Buffer

#49

Post by kylebarron »

paddy_m wrote: post #44

Somewhat related, who has played with performant JS dataframe libraries? I mostly want to serialize from pandas/polars to JS in a fast way, with some very minor selection and indexing operations JS side. Ideally I could find a library that does dataframe like operations from reading base64 encoded Buffers/Arrays into their JS objects, then doing the right things. I'm investigating https://arrow.apache.org/docs/js/ ht…

nodejs-polars is node-specific and uses native FFI. polars can be compiled to Wasm but doesn't yet have a js API out of the box.

As for the fastest way to serialize data to Pandas data to the browser, you should use Parquet; it's the fastest to write on the Python side and read on the JS side, while also being compressed. See https://github.com/kylebarron/parquet-wasm (full disclosure, I wrote this)

Re: Goodbye, Node.js Buffer

#50

Post by iso8859-1 »

jongjong wrote: post #12

TBH I prefer Buffer as an abstraction. Too bad it didn't make it into native JS standard. I can't think of many use cases in JS land were Uint8Array, Uint16Array, Uint32Array, Int8Array would be absolutely necessary. Seems more useful for perf optimizations. For WebAssembly? Surely a plain array of numbers can be used in most cases. The main use case for Buffer IMO is to convert between different formats like Base64,…

> For WebAssembly? Surely a plain array of numbers can be used in most cases.

So a list of floats? No! Let's not use floats everywhere just because JavaScript does.

Post reply on HN