41–50 of 75 posts
Earlier quoted context omitted.
People still write in languages that force you to manage your own memory. Once performance starts to matter (either due to scale or time requirements) abstractions always have tradeoffs you can't accept.
So then, how can garbage collection ever be solved if it’s a trade-off
As long as compute is a meaningful percentage of spend, the trade off will matter.
Earlier quoted context omitted.
So then, how can garbage collection ever be solved if it’s a trade-off
And how can backends be abstracted away if there is a trade off? As long as compute is a meaningful percentage of spend, the trade off will matter.
Earlier quoted context omitted.
> Because they do things. With their hands. That no one else does That's only true of surgeons :) What if your specialty is nonsurgical (internal medicine, pediatrics, psychiatry, etc)?
Not even true of all surgeons, the ones that make the most money use machines to work on things their hands couldn't do
Earlier quoted context omitted.
And how can backends be abstracted away if there is a trade off? As long as compute is a meaningful percentage of spend, the trade off will matter.
Right. So what does it look like for garbage collection to be solved? You’re saying it’s not ever possible
Is the code written by the deepseek model? I should probably give up on being a software engineer if it is.
Earlier quoted context omitted.
Not even true of all surgeons, the ones that make the most money use machines to work on things their hands couldn't do
pathologists are some of the highest paid doctors and they are right in the crosshairs of what AI is getting better at performing.
One thing I found peculiar is that for the GraySort benchmark it dispatches to Polars by default to do the actual sorting, not DuckDB: https://github.com/deepseek-ai/smallpond/blob/ed112db42af4d0... .
https://github.com/deepseek-ai/smallpond/blob/ed112db42af4d0...
We are seeing more and more specialized query engines. This is a query engine specialized for training pipelines. It is not general purpose - it is for providing batches of training data at workers. It uses Ray for parallelization. The kind of queries you need are random reads (to implement shuffling across epochs), arrow support (zero copy to Pandas DataFrames), and efficient checkpointing.
Is the code written by the deepseek model? I should probably give up on being a software engineer if it is.
Give up and become what? Most white collar jobs will be automated in the coming years. You think doctors' jobs are safe?
Earlier quoted context omitted.
Yes, doctors are safe. Because they do things. With their hands. That no one else does.
> Because they do things. With their hands. That no one else does That's only true of surgeons :) What if your specialty is nonsurgical (internal medicine, pediatrics, psychiatry, etc)?