Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GigaToken: ~1000x faster Language model tokenization

github.com

Post reply on HN
Print view View original

81–90 of 137 posts

Re: GigaToken: ~1000x faster Language model tokenization

#81

Post by asdf88990 »

cschmidt wrote: post #80
asdf88990 wrote: post #79

Earlier quoted context omitted.

[flagged]

I’m not sure why you think this is ai slop. I work on tokenization research full time. My name is Craig Schmidt and I have a number of papers in the field. This researcher has done some very impressive work and I’m trying to defend him from the HN dismissive hoards. There is a serious research community on tokenization, and we are quite interested in this work.

Which part of “doesn’t seem to be aislop signs” made you think I believe your comment was aislop?

As I said, It just might be the low information density approach to talking recently that seems off to me.

To explain a bit more, I kept reading and waiting for the penny drop but nothing.

“I cloned your repo” — okay then what happened? Nothing? I put my shoes on this morning.

“I know the tokrniziation community wants to absord tje lessons” okay? Tell me the point please! I know the AI community wants to understand the universe.

“These are useful ideas”. Yes.

Of course, you could argue that my comment falls in the same category in the sense that I am not actually contributing to the topic at hand but I am peeved with all low the information noise.

Re: GigaToken: ~1000x faster Language model tokenization

#82

Post by cs702 »

cschmidt wrote: post #76

Can I say this seems to be fantastic work. I cloned your repo earlier today after seeing it on the tokenization discord. I know everyone in the tokenization community wants to absorb the lessons of how you got such a speedup. The caching and replacing the regex for pretokenization seem like generally useful ideas. And screw all the 0.1% haters on here, this is great stuff.

That is my reaction too. It looks like great work!

Valuable not only for inference, but for training too (think proprietary datasets).

I would add, a single individual did this.

One person can make a difference :-)

Re: GigaToken: ~1000x faster Language model tokenization

#84

Post by jaggederest »

dataflow wrote: post #71
jaggederest wrote: post #48

Earlier quoted context omitted.

To concur it's "latency critical", not "performance critical", people often confuse those two - optimize it all, but especially the chained critical path latency!

Latency isn't performance? Maybe you mean "not throughput-critical"?

Just to clarify, latency is one form of performance, and a separate thing to optimize from total resource usage in more classic "performance critical" situations. That performance might be energy, space, or other dimensions besides latency. It might also be something like reliability, accuracy, precision, or even the very human factors like simplicity, modifiability, and visibility.

Heck, even latency alone you can just reduce the standard deviation and get smoother flows. Little's law is a great callout here too, one of my favorite computer science principles.

Re: GigaToken: ~1000x faster Language model tokenization

#85

Post by victor106 »

cschmidt wrote: post #76

Can I say this seems to be fantastic work. I cloned your repo earlier today after seeing it on the tokenization discord. I know everyone in the tokenization community wants to absorb the lessons of how you got such a speedup. The caching and replacing the regex for pretokenization seem like generally useful ideas. And screw all the 0.1% haters on here, this is great stuff.

> tokenization discord

How can I join this? Sounds interesting

Re: GigaToken: ~1000x faster Language model tokenization

#86

Post by marcelroed »

cschmidt wrote: post #76

Can I say this seems to be fantastic work. I cloned your repo earlier today after seeing it on the tokenization discord. I know everyone in the tokenization community wants to absorb the lessons of how you got such a speedup. The caching and replacing the regex for pretokenization seem like generally useful ideas. And screw all the 0.1% haters on here, this is great stuff.

Thanks for the kind words, Craig! I'm planning to do a technical writeup+paper and a presentation video on the project in the near future. Will make sure to share it with the Discord!

Re: GigaToken: ~1000x faster Language model tokenization

#87

Post by glenngillen »

victor106 wrote: post #85
cschmidt wrote: post #76

Can I say this seems to be fantastic work. I cloned your repo earlier today after seeing it on the tokenization discord. I know everyone in the tokenization community wants to absorb the lessons of how you got such a speedup. The caching and replacing the regex for pretokenization seem like generally useful ideas. And screw all the 0.1% haters on here, this is great stuff.

> tokenization discord How can I join this? Sounds interesting

also subscribing to this!

Re: GigaToken: ~1000x faster Language model tokenization

#88

Post by gghfez »

cschmidt wrote: post #76

Can I say this seems to be fantastic work. I cloned your repo earlier today after seeing it on the tokenization discord. I know everyone in the tokenization community wants to absorb the lessons of how you got such a speedup. The caching and replacing the regex for pretokenization seem like generally useful ideas. And screw all the 0.1% haters on here, this is great stuff.

>the tokenization discord

Could I join this?

Re: GigaToken: ~1000x faster Language model tokenization

#89

Post by njjkkjjdhs »

marcelroed wrote: post #86
cschmidt wrote: post #76

Can I say this seems to be fantastic work. I cloned your repo earlier today after seeing it on the tokenization discord. I know everyone in the tokenization community wants to absorb the lessons of how you got such a speedup. The caching and replacing the regex for pretokenization seem like generally useful ideas. And screw all the 0.1% haters on here, this is great stuff.

Thanks for the kind words, Craig! I'm planning to do a technical writeup+paper and a presentation video on the project in the near future. Will make sure to share it with the Discord!

[dead]
Post reply on HN