Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malloc Challenge

vicsydev.blogspot.com

Post reply on HN
Print view View original

31–40 of 109 posts

Re: Malloc Challenge

#31

Post by _RPM »

greenshackle2 wrote: post #30
_RPM wrote: post #29

Earlier quoted context omitted.

What the hell is C-Like Python?

Python that looks like C? It's not a thing, I just made it up. A subset of Python that is not too painful to rewrite in C. No classes, I use namedtuples as structs, etc. I'm not sure it's a sane thing to do, arguably I should just write the thing in C in the first place, but I have more experience with Python so it's still easier to me.

Python does not look like C. The first thing I can think of is using traditional loop counters instead of the for..in construct.

Re: Malloc Challenge

#32

Post by pheo »

Let's all just agree to use Perl. It's just dynamic, functional, OOPy C isn't it?

J/K. I think this is an interesting problem in that its a sandbox for allocation and GC in pretty much any dynamic interpreter's implementation. My qualm is that it would be "easy" to tune for the test. Consider the difference between dynamic blocks of a small but fixed size, getting alloc'd/freed in an asynchronous way (a network stack?) versus a pool of variable byte length strings getting shuffled around (a key/value store?). Those are simple, but drastically different, strategies for your heap. There won't be a "best" answer besides the limits of your problem domain.

Re: Malloc Challenge

#34

Post by greenshackle2 »

_RPM wrote: post #31
greenshackle2 wrote: post #30

Earlier quoted context omitted.

Python that looks like C? It's not a thing, I just made it up. A subset of Python that is not too painful to rewrite in C. No classes, I use namedtuples as structs, etc. I'm not sure it's a sane thing to do, arguably I should just write the thing in C in the first place, but I have more experience with Python so it's still easier to me.

Python does not look like C. The first thing I can think of is using traditional loop counters instead of the for..in construct.

That's a minor detail. I'm talking about writing in a procedural style.

Rewriting a for..in as a for with counters is relatively trivial, compared to re-implementing a complex class hierarchy in C.

You can write procedural Python, and it will look kinda like C if you squint. Cython even lets you use C types directly, with a simple syntax. As I said, maybe not the sanest thing to do, everything is an object in Python so you can never really get away from it's object-oriented nature.

Re: Malloc Challenge

#35

Post by naasking »

codr4life wrote: post #28
sparky_ wrote: post #22

Earlier quoted context omitted.

Somebody out there is doing this in production. I guarantee it.

Why wouldn't they? If it turns out to be a good solution for the problem they're trying to solve? There's nothing wrong with coming up with your own solutions, that's why we got brains instead of answering machines.

It is a problem if it's not properly documented. This system will probably outlast their time at that company, so the next poor shmuck is left to figure out what that program is doing.

Re: Malloc Challenge

#37

Post by codr4life »

pheo wrote: post #32

Let's all just agree to use Perl. It's just dynamic, functional, OOPy C isn't it? J/K. I think this is an interesting problem in that its a sandbox for allocation and GC in pretty much any dynamic interpreter's implementation. My qualm is that it would be "easy" to tune for the test. Consider the difference between dynamic blocks of a small but fixed size, getting alloc'd/freed in an asynchronous way (a network stack…

Been there, done that. Even went to YAPC EU in Pisa and had a whiff of Larry. Not for me, that's all I can say. It's too loose, too much shooting from the hip. There's a lot of good ideas in there though.

Agreed. Which is why the 'one size fits all' approach might not be the best way to go. The main reason I decided to launch the challenge, and encourage a more combinatory approach with local special purpose allocators.

Re: Malloc Challenge

#38

Post by throwaway4891a »

See also:

- http://locklessinc.com/benchmarks_allocator.shtml ($, use as a minimum performance target)

- http://www.nedprod.com/programs/portable/nedmalloc/

- http://phk.freebsd.dk/pubs/malloc.pdf [PDF] (phkmalloc)

- https://github.com/gperftools/gperftools (tcmalloc)

- https://github.com/ivmai/bdwgc/blob/master/malloc.c

- https://github.com/jemalloc/jemalloc

- http://gee.cs.oswego.edu/dl/html/malloc.html (dlmalloc)

Re: Malloc Challenge

#39

Post by codr4life »

anaccountwow wrote: post #12
gbarboza wrote: post #8

All past 15-213 students go in search of their malloc lab solution.

Current 15-213 student here in the middle of malloc lab, my current solution is probably worse than anything in that link (left unclicked).

Give it a spin, you might learn something. Your class is probably focusing on general purpose, system level allocators; a much thornier problem without any really good answers. As How to Solve It states; if you can't solve the given problem, try to solve a simpler version of the problem.

Re: Malloc Challenge

#40

Post by codr4life »

otabdeveloper wrote: post #33

A malloc benchmark that doesn't measure multithreaded performance is worse than useless.

Only if you insist on clinging to a general purpose, system level perspective on memory allocation. No amount of thinking and reasoning about these issues is useless.
Post reply on HN