Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using mmap to make LLaMA load faster

justine.lol

Post reply on HN
Print view View original

91–100 of 186 posts

Re: Using mmap to make LLaMA load faster

#91

Post by oceanplexian »

candrewlee14 wrote: post #14

jart is a genius. What they’ve done with Blink, Cosmopolitan C, Redbean, and now llama.cpp is incredible. It gives me hope for the future of systems/low-level programming.

I suspect you're trolling but Jart is not a genius for co-authoring a 300 line change to use mmap(), this is a pretty typical system engineering optimization that goes on in big companies, which is usually framed as a performance improvement for certain use cases with pros and cons, not "Hacker news front page 100X revolutionary performance improvement".

Re: Using mmap to make LLaMA load faster

#92

Post by dimator »

candrewlee14 wrote: post #14

jart is a genius. What they’ve done with Blink, Cosmopolitan C, Redbean, and now llama.cpp is incredible. It gives me hope for the future of systems/low-level programming.

holy cow, her website is enough to give me imposter syndrome https://justine.lol/

Re: Using mmap to make LLaMA load faster

#93

Post by tedunangst »

chad1n wrote: post #80
vore wrote: post #53

Earlier quoted context omitted.

jart is a woman, for what it's worth.

I just presented the drama and I referred to him as `he` as a developer, my teammates are mostly male, so I usually say `he`.

Do you also refer to your female teammates as he?

Re: Using mmap to make LLaMA load faster

#96

Post by HopenHeyHi »

MuffinFlavored wrote: post #70
mhh__ wrote: post #52

I've seen this on my HN feed about 5 times now, twitter twice. Why? (Author's blog is acceptable as they did the change, but the other 4?) Is this AI people learning that mmap exists? People simping cos' there a Justine involved?

[flagged]

No? I mean yes, probably will get downvotes from the usual crowd.

There is as a matter of fact some of that explicitly going on in the GitHub issue, which derailed the conversation.

Such are the woes of being an internet micro celebrity I guess. Truly we live in the stupidest timeline.

Re: Using mmap to make LLaMA load faster

#98

Post by baq »

csmpltn wrote: post #62
astrange wrote: post #38

Earlier quoted context omitted.

Using mmap is basic systems programming knowledge and not "incredible" even though it's a useful contribution.

Glad to see we're slowly coming to the realization that wrangling NumPy and Pandas doesn't actually count as Software Engineering.

it's as much engineering as is gluing mmap, recv and send... there are people who can do that and can't piece together a dataframe pipeline, no need to be passive agressive here

Re: Using mmap to make LLaMA load faster

#99

Post by tedunangst »

pantalaimon wrote: post #39
NickGerleman wrote: post #6

Worth pointing out, there has been quite a bit of contention around this change, both technical, and some accusations of plagiarism/miscrediting here. https://github.com/ggerganov/llama.cpp/pull/711

Is mmap really that broken on Windows? Or is the poster just confused that the data stays in the page cache? But that’s what the page cache does - that memory will be used for other things if needed, but if the memory is not needed it might as well keep the old data in cache.

There's a weird breed of programmer who only wants to see the free memory column in top be maximized. I bought all this RAM and I want to make sure none of it is used in case I want to use it later.

Re: Using mmap to make LLaMA load faster

#100

Post by dekhn »

timpani wrote: post #94
vore wrote: post #53

Earlier quoted context omitted.

jart is a woman, for what it's worth.

[flagged]

This sort of statement really don't contribute anythign to the discussion and in fact greatly distracts from the technical content. We don't need to hear your opinions about trans people in this thread.
Post reply on HN