Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating away from Rust

deadmoney.gg

Post reply on HN
Print view View original

731–740 of 799 posts

Re: Migrating away from Rust

#731

Post by Animats »

josephg wrote: post #430
_bin_ wrote: post #297

Earlier quoted context omitted.

I saw a good talk, though I don't remember the name, that went over the array-index approach. It correctly pointed out that by then, you're basically recreating your own pointers without any of the guarantees rust, or even C++ smart pointers, provide.

> It correctly pointed out that by then, you're basically recreating your own pointers without any of the guarantees rust, or even C++ smart pointers, provide. I've gone back and forth on this, myself. I wrote a custom b-tree implementation in rust for a project I've been working on. I use my own implementation because I need it to be an order-statistic tree, and I need internal run length encoding. The original vers…

> What it doesn't protect you from is use-after-free bugs.

Yes. I've found that problem in index-allocated code.

Also, when you do this, you need an allocator for the indexes. I've found bugs in those.

Re: Migrating away from Rust

#732

Post by AIPedant »

I realize there were bigger problems, but this makes me very sad:

  Learning - Over the past year my workflow has changed immensely, and I regularly use AI to learn new technologies, discuss methods and techniques, review code, etc. The maturity and vast amount of stable historical data for C# and the Unity API mean that tools like Gemini consistently provide highly relevant guidance. While Bevy and Rust evolve rapidly - which is exciting and motivating - the pace means AI knowledge lags behind, reducing the efficiency gains I have come to expect from AI assisted development. This could change with the introduction of more modern tool-enabled models, but I found it to be a distraction and an unexpected additional cost.
In 2023 I wondered if LLM code generation would throttle progress in programming language design. I was particularly thinking about Idris and other dependently-typed languages which can do deterministically correct code generation. But it applies to any form of language innovation: why spend time learning a new programming language that 100% reliably abstracts boilerplate away, when an LLM can 95% reliably slop the boilerplate? Some people (me) will say that this is unacceptably lazy and programmers should spend time reading things, the other will point to the expected value of dev costs or whatever. Very depressing.

Re: Migrating away from Rust

#733

Post by koakuma-chan »

palata wrote: post #723
koakuma-chan wrote: post #690

Earlier quoted context omitted.

Those are interpreted.

You are one of those people stuck in the 90s, then. Take 5 min to read about JIT and AOT compilation.

What point are you trying to make? V8 has a JIT compiler too, does that make JavaScript a compiled language?

Re: Migrating away from Rust

#734

Post by neonsunset »

Groxx wrote: post #725
neonsunset wrote: post #724

Earlier quoted context omitted.

And so the broken record takes another turn...

When the broken record keeps predicting the future accurately, it's not claiming to be the next Nostradamus. History just repeats pretty consistently.

By this logic we should drop Golang immediately then because tomorrow Google will either kill it or they will add unremovable telemetry that uploads all of your code directly to AdSense.

Re: Migrating away from Rust

#735

Post by cantrecallmypwd »

fc417fc802 wrote: post #376
cantrecallmypwd wrote: post #330

Earlier quoted context omitted.

History lesson for the cheap seats in the back: Compressed tars are terrible for random access because the compression occurs after the concatenation and so knows nothing about inner file metadata, but it's good for streaming and backups. Uncompressed tars are much better for random access. (Tar was a used as a backup mechanism to tape (tape archive).) Zips are terrible for streaming because their metadata is stored…

> compressing files and then taring them Just use squashfs if that is the functionality that you need.

That works too if it's available.

Re: Migrating away from Rust

#736

Post by josephg »

variadix wrote: post #702
akoboldfrying wrote: post #513

Earlier quoted context omitted.

> Recently I rewrote the b-tree to simply use a vec of internal nodes Doesn't this also require you to correctly and efficiently implement (equivalents of C's) malloc() and free()? IIUC your requirements are more constrained, in that malloc() will only ever be called with a single block size, meaning you could just maintain a stack of free indices -- though if tree nodes are comparable in size to integers this increa…

Depends on if you need to allocate/deallocate nodes. If you construct the tree once and don’t modify it thereafter you don’t need to. If you do need to modify and alloc/dealloc nodes you can use a bitmap to track free/occupied slots which is very fast (find first set + bitmanip) and has minuscule overhead even for integer sized elements.

Yeah, or just store all freed nodes in a linked list. Eg, have a pointer / index from the root to the first unused (free) node, and in that node store a pointer to the next one and so on. This is pretty trivial to implement.

In my case, inserts and read operations vastly outnumber deletes. So much so that in all of my testing, I never saw a leaf node which could be freed anyway. (Leaves store ~32 values, and there were no cases where all of a leaf's values actually get deleted). I decided to just leak nodes if it ever happens in real life.

The algorithm processes data in batches then frees everything. So worst case, it just has slightly higher peak memory usage while processing. A fine trade in this case given it let me remove ~200 lines of code - and any bugs that might have been lurking in them.

Re: Migrating away from Rust

#737

Post by josephg »

jstimpfle wrote: post #580
josephg wrote: post #430

Earlier quoted context omitted.

> It correctly pointed out that by then, you're basically recreating your own pointers without any of the guarantees rust, or even C++ smart pointers, provide. I've gone back and forth on this, myself. I wrote a custom b-tree implementation in rust for a project I've been working on. I use my own implementation because I need it to be an order-statistic tree, and I need internal run length encoding. The original vers…

Having gone full-in on this approach before, with some good success, it still feels wrong to me today. Contiguous storage may work for reasonable numbers of elements, but it's potentially blocking a huge contiguous chunk of address space especially for large numbers of elements. I probably say this because I still have to main 32-bit binaries (only 2G of address space), but it can potentially be problematic even on 6…

> Having gone full-in on this approach before, with some good success, it still feels wrong to me today. Contiguous storage may work for reasonable numbers of elements, but it's potentially blocking a huge contiguous chunk of address space especially for large numbers of elements.

That makes sense. If my btree was gigabytes in size, I might rethink the approach for a number of reasons. But in my case, even for quite large input, the data structure never gets more than a few megabytes in size. Thats small enough that resizing the vec has a negligible performance impact.

It helps that my btree stores its contents using lossless internal run-length encoding. Eg, if I have values like this:

    {key: 5, value: 'a'}
    {key: 6, value: 'a'}
    {key: 7, value: 'a'}
Then I store them like this:

    {key: [5..8), value: 'a'}
In my use case, this compaction decreases the size of the data structure by about 20x. There's some overhead in joining and splitting values - but its easily worth it.

Re: Migrating away from Rust

#738

Post by at_compile_time »

bunderbunder wrote: post #161
Pet_Ant wrote: post #132

Earlier quoted context omitted.

> Oftentimes the data gets organized in parallel arrays ( https://en.wikipedia.org/wiki/Parallel_array ) instead of in collections of structs. This can save a lot of memory (because the data gets packed more densely) be more cache-friendly, and makes it much easier to make efficient use of SIMD instructions. That seems like something that could very easily be turned into a compiler optimisation and enabled with somet…

Meh. I've tried "SIMD magic wand" tools before, and found them to be verschlimmbessern. At least on the scientific computing side of things, having the way the code says the data is organized match the way the data is actually organized ends up being a lot easier in the long run than organizing it in a way that gives frontend developers warm fuzzies and then doing constant mental gymnastics to keep track of what the…

> verschlimmbessern

Thank you for this delightful word.

Re: Migrating away from Rust

#739

Post by at_compile_time »

anticrymactic wrote: post #91
koakuma-chan wrote: post #60

Earlier quoted context omitted.

The fact you need a usize specifically to index an array (and most collections) is pretty annoying.

This could be different in game dev, but in the last years of writing rust (outside of learning the language) I very rarely need to index any collection. There is a very certain way rust is supposed to be used, which is a negative on it's own, but it will lead to a fulfilling and productive programming experience. (My opinion) If you need to regularly index something, then you're using the language wrong.

On the contrary, I find indices to be the most natural way to represent anything that resembles a graph in Rust. They allow you to sidestep the usual issues that arise with ownership and borrowing, particularly with mutability, by handing ownership to the collection and using indices to allow nodes to refer to one another. It's delightfully simple compared to the mess of Arc and RefCell that tends to result when one tries to apply patterns from languages that leave "shared XOR mutable" as the programmer's responsibility. That's not to say that Vec and usize are appropriate for the task, but Rust's type system can be used to do a lot better.

Re: Migrating away from Rust

#740

Post by at_compile_time »

lynndotpy wrote: post #629
bionhoward wrote: post #395

Earlier quoted context omitted.

Sounds like a good use of Num [1] https://docs.rs/num-traits/latest/num_traits/trait.Num.html

Please correct me if I'm wrong, but I don't think this would let me, say, pass an i32 returned from one method directly as an f64 argument in another method.

No, it would not. Even conversions using "as" are discouraged in favor of conversion traits such as From and TryFrom. Rust's goals of being explicit and correct are at odds with people wanting things to be immediately simple and easy to use.
Post reply on HN