Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malloc Challenge

vicsydev.blogspot.com

Post reply on HN
Print view View original

71–80 of 109 posts

Re: Malloc Challenge

#71

Post by userbinator »

greenshackle2 wrote: post #21

>[libc4life] is aiming for simplicity and leverage; and it makes a real effort to get there by playing on C's strengths, rather than just inventing yet another buggy Lisp. Ouch, right in the feels, I've been working on https://buildyourownlisp.com in my spare time. (EDIT: I was looking for a name for the repo, YABLisp it is.) > coding in C is a welcome therapy after seemingly wasting years exploring various ways of p…

But I wonder what hope there is for people like us in the industry, which seems to be moving ever further away from this type of programming.

C is still very popular (along with Asm) in embedded systems. ARM cores and large memories are certainly getting very cheap, but still not at the level of many of the 8 and 16-bit microcontrollers.

I find OOP often overused too, but it can be genuinely useful in certain situations where there is a very strong association between some data and the operations on it. For this reason, I tend to use a subset of C++ in a style that would probably enrage a lot of the "C++ is not C" advocates.

Re: Malloc Challenge

#72

Post by ben_bai »

wfunction wrote: post #65
codr4life wrote: post #46

Earlier quoted context omitted.

The pool reference allocator does just that internally. It prefixes each allocation with a block containing the size among other things. Either base your implementation on top of that or take the idea and run with it.

You totally missed the point of my question. I was NOT asking "I have an allocator that doesn't store the buffer size; how can I use it?" I was asking, "I don't think an allocator should need to store the buffer size internally; why not formulate the challenge so that the block size doesn't need to be stored?"

I would not do that. Userspace Apps have way to many ways to screw up memory managment already. Allowing them to

    buf=malloc(128);
    free(buf, 256);
seems dangerous, if free can't check the size.

But kernels sometimes do just that ( free(ptr, size) ), for performance reasons and because "kernel writers know what they are doing".

Re: Malloc Challenge

#73

Post by mathgenius »

codr4life wrote: post #43
throwaway4891a wrote: post #38

See also: - http://locklessinc.com/benchmarks_allocator.shtml ($, use as a minimum performance target) - http://www.nedprod.com/programs/portable/nedmalloc/ - http://phk.freebsd.dk/pubs/malloc.pdf [PDF] (phkmalloc) - https://github.com/gperftools/gperftools (tcmalloc) - https://github.com/ivmai/bdwgc/blob/master/malloc.c - https://github.com/jemalloc/jemalloc - http://gee.cs.oswego.edu/dl/html/malloc.html (dlmalloc)

Thank you, but that's not the problem I'm solving. Any one of those could be used to feed any implementation from the challenge. The idea I'm pushing is using local knowledge to customize memory management, instead of trying to find the perfect one-size-fits-all solution.

Aha, I see.. Very nice. Like the following implementation of free() as commonly used in HFT:

void free(void *mem) {}

Re: Malloc Challenge

#74

Post by to3m »

nullc wrote: post #57
EdSharkey wrote: post #41

I recall reading in the last year or two a recount of how a game developer got their PS3 engine to run at 30 or 60Hz framerate by aggressive triple-buffering of their scenes. One of the interesting bits about the article was their memory allocation scheme. Each game frame they'd allocate a single huge memory pool and then allocate from it by simply incrementing a pointer into the pool. I think this is what you descri…

> a single huge memory pool and then allocate from it by simply incrementing a pointer into the pool Maybe a register could be used to keep track of the current pointer into this pool, and every time you called into a new function the call could increment the pointer enough for all the usage in a function, and when the function returns it could set it back, automatically deallocating the usage with almost no cost at…

But not popping the last allocation when the function returns is kind of the whole point... it is (or certainly was...) common to double-buffer these, so that one frame's data is built up, and then consumed by another thread/the GPU/etc. while the corresponding data for the next frame is being filled in.

Re: Malloc Challenge

#76

Post by conjectures »

userbinator wrote: post #71
greenshackle2 wrote: post #21

>[libc4life] is aiming for simplicity and leverage; and it makes a real effort to get there by playing on C's strengths, rather than just inventing yet another buggy Lisp. Ouch, right in the feels, I've been working on https://buildyourownlisp.com in my spare time. (EDIT: I was looking for a name for the repo, YABLisp it is.) > coding in C is a welcome therapy after seemingly wasting years exploring various ways of p…

But I wonder what hope there is for people like us in the industry, which seems to be moving ever further away from this type of programming. C is still very popular (along with Asm) in embedded systems. ARM cores and large memories are certainly getting very cheap, but still not at the level of many of the 8 and 16-bit microcontrollers. I find OOP often overused too, but it can be genuinely useful in certain situati…

I've noticed this, and also that the typical salaries for embedded systems developers seems to be lower than for hotness.js developers :(

Am I wrong? Hope so.

Re: Malloc Challenge

#78

Post by greenshackle2 »

userbinator wrote: post #71
greenshackle2 wrote: post #21

>[libc4life] is aiming for simplicity and leverage; and it makes a real effort to get there by playing on C's strengths, rather than just inventing yet another buggy Lisp. Ouch, right in the feels, I've been working on https://buildyourownlisp.com in my spare time. (EDIT: I was looking for a name for the repo, YABLisp it is.) > coding in C is a welcome therapy after seemingly wasting years exploring various ways of p…

But I wonder what hope there is for people like us in the industry, which seems to be moving ever further away from this type of programming. C is still very popular (along with Asm) in embedded systems. ARM cores and large memories are certainly getting very cheap, but still not at the level of many of the 8 and 16-bit microcontrollers. I find OOP often overused too, but it can be genuinely useful in certain situati…

I work on a modular business application at work, and OOP is useful in places. I don't hate it, but it can be kinda clunky.

We have an embedded systems team at work but they barely have enough work to keep the current small dev team busy.

Re: Malloc Challenge

#79

Post by Beltiras »

greenshackle2 wrote: post #34
_RPM wrote: post #31

Earlier quoted context omitted.

Python does not look like C. The first thing I can think of is using traditional loop counters instead of the for..in construct.

That's a minor detail. I'm talking about writing in a procedural style. Rewriting a for..in as a for with counters is relatively trivial, compared to re-implementing a complex class hierarchy in C. You can write procedural Python, and it will look kinda like C if you squint. Cython even lets you use C types directly, with a simple syntax. As I said, maybe not the sanest thing to do, everything is an object in Python…

Python is not really object oriented. To get information hiding you have to wrap your class in a closure and that closure has to track the individual instances. It's not clean code and it's horribly inefficient.

Python is also not a functional language.

Python provides language structures that allow you to use both styles of programming OOP and functional as-if-it-was.

Re: Malloc Challenge

#80

Post by mauvehaus »

conjectures wrote: post #76
userbinator wrote: post #71

Earlier quoted context omitted.

But I wonder what hope there is for people like us in the industry, which seems to be moving ever further away from this type of programming. C is still very popular (along with Asm) in embedded systems. ARM cores and large memories are certainly getting very cheap, but still not at the level of many of the 8 and 16-bit microcontrollers. I find OOP often overused too, but it can be genuinely useful in certain situati…

I've noticed this, and also that the typical salaries for embedded systems developers seems to be lower than for hotness.js developers :( Am I wrong? Hope so.

My experience, coming as somebody who has experience on the large end of embedded (100's of MHz, 100's of MB RAM, MMU, running Linux), albeit a while ago, is that it's hard to break back into it if you've been out of the community for a while.

The last time I was looking for a job, I had been doing non-embedded, including kernel and low-level for 8 years or so, and I had the darnedest time finding embedded positions to apply for, let alone get calls back from.

Startups that do embedded seem to mostly get the embedded done in the MVP stage, and by the time they're looking to grow, they've got a (small) team already doing that work.

In fairness, this is partially self-inflicted; iRobot has been on the monthly whoishiring threads for as long as I've been looking for jobs in the Boston area, but I decided long ago that I'd never work in defense (or finance).

Recently, I've added to that list "companies whose business model is selling their users' information" (many of them these days), and "companies whose entire sales pitch plays on your fears" (a recruiter tried to put me in touch with a company that does baby monitors that measure some basic health parameters, and their promotional video was appalling to me).

I kind of stuck a toe across the first line, and tried talking to somebody Fitbit-esque, but never got a call back. I also wound up with a connection into an electronic paper company, but that went cold after submitting a resume. Ditto a company that does automotive suspension stuff.

I realize that being out of the community for a while makes it hard to get consideration if there are other candidates who have current experience, so I started a personal project building a clock out of VFD tubes, and added a github link to my resume before submitting several of those. That didn't seem to help.

I got hired doing non-embedded before I finished the clock project. I'm at the point of needing to learn EAGLE, lay out boards and get them made. That was a year and a bit ago; right now I've got other projects that take up my weekend time.

To respond directly to your point, I'm willing to bet that there are two factors at work: 1. Supply and demand. If demand exceeded supply, I would think that I'd at least be able to get a call back with my background. 2. Hardware is a hard row to hoe. There's a lot less capital invested in a hotness.js project that flops, and it's a lot easier to scale if it takes off. No, it's not necessarily easy, but there's a wide gap between "provision more instances on EC2 and fix the bottlenecks in the architectures" and "the factory is already at capacity and the supplier for the key part is backordered for 3 months". The money that doesn't need to go to covering those risks can go into your pockets.

Post reply on HN