Viewing profile — thntk

The model architecture only uses and cites pre-2023 techniques from the GPT-2 and GPT-3 era. Probably they intentionally tried to use the most bare transformers architecture possib…

Anyone know if it can run training/fine-tuning and not just 4-bit inference? Does it support mixed precision training with either BF16 or FP16?

It sounds like your startup is doing great, good luck.

Have you tried hiring or rotating internal people just for solving specific (management) tasks? It is like the "just-in-time" style in Japanese corps.

It's not founder mode or manager mode. I think it is just about effective management. When starting up, the founder needs to (1) know what should be done, (2) be able to do it them…

Isn't it the technological cycle in IT? We have seen PC softwares in the 90s, then web apps, then mobile apps, and now AI services. Except that currently AI is immature, so compani…

Anyone know what caused the very big performance jump from Large1 to Large2 in just a few months? Besides, parameter redundancy seems evidenced. Front-tier models used to be 1.8T, …

When Zuck said spy can easily steal models, I wonder how much of it comes from experiences. I remember they struggled to train OPT not long ago. On a more serious note, I don't rea…

Correct me if I'm wrong, my impression is that 3.1 is a better fine-tuned variant of base 3.0 with extensive use of synthetic data.

I've seen such articles more and more recently. In the past, when people had a vague idea, they had to do research before writing. During this process, they often realized some fla…

We knew high quality data can help as evidenced by the \Phi models. However, this alone can never eliminate hallucination because data can never be both consistent and complete. Mo…

Besides the burden of knowledge and the flaws of academic funding, another factor to explain the slowdown in scientific progress is: the world has more things to be entertained wit…

Oh, that is so ridiculous that I have to come back and LOL.

What does porn have to do with free speech and privacy?