Viewing profile — joshhart

Discriminative has a meaning in machine learning that I think is relevant here. There are "generative" models like LLMs that are learning joint probabilities P(X, Y) and "discrimin…

According to the article, the major effect is they are catching and prosecuting cheaters with a secondary effect that students are over-relying on them for homework or practice, an…

Fireworks will serve them for $1.74 / $0.14 / $3.48. That's input / cached input / output. https://fireworks.ai/models/deepseek-ai/deepseek-v4-pro . Call it about a third the price…

If you have good ideas that have a nice return on investment and leverage existing skills, sure. If you don’t have good opportunity laying around, best for the business to switch t…

I thought this wasn't viable due to cooling requirements - how do you cool massive amounts of compute when the only option is to radiate it into space - nothing to convect it with?…

Vitamin D toxicity is absolutely real, causes hypercalcemia, and can occur even at the 4,000 IU dose. I would really recommend you be getting regular bloodwork done if you go beyon…

Huh. The standard in your case is to measure waist circumference if BMI is high. Did no doctor do that? As long as you are below 40” or 37” if Asian you are considered good to go.

This is super awesome, but how in the world did they come up with a name "Nano Banana Pro"? It sounds like an April Fools joke.

I believe it's intended to convince the audience they are experts, that this type of thing is dangerous to a business, and they are the ones doing the most to prevent it. There is …

So the inference speed at low to medium usage is memory bandwidth bound, not compute bound. By “forecasting” into the future you do not increase the memory bandwidth pressure much …

A single node with GPUs has a lot of FLOPs and very high memory bandwidth. When only processing a few requests at a time, the GPUs are mostly waiting on the model weights to stream…

This is pretty exciting. Now an organization could produce an open weights mixture of experts model that has 8-15b active parameters but could still be 500b+ parameters and it coul…

Hi, Yes you can. The community creates quantized variants of these that can run on consumer GPUs. A 4-bit quantization of LLAMA 70b works pretty well on Macbook pros, the neural en…

The benchmarks compare it favorably to GPT-4-turbo but not GPT-4o. The latest versions of GPT-4o are much higher in quality than GPT-4-turbo. The HN title here does not reflect wha…

Hi, I run the model serving team at Databricks. Usually you run regex filters, LLAMA Guard, etc on chunks at a time so you are still streaming but it's in batches of tokens rather …

[flagged]

I spent 12 years at LinkedIn. Sadly, it's not even close to the engineering org it used to be. The era where Kevin Scott led engineering was a really good one in comparison.

At Databricks we have an LLM that is fine-tuned to do the problem you raise - https://www.databricks.com/blog/announcing-public-preview-ai... Many customers like it a lot. Although…

Makes sense, CPUs and memory sizes aren’t growing that fast anymore. But I’m sure they are spending a ton on TPUs/GPUs, the article is clear on very high capex

If you are making many requests in batch this works ok because you can shuffle the next layer in while the current one is processing a set of matrix multiplies. This takes it from …

LinkedIn was already very FCF positive. They tightly managed margins to get to net income positive (account for dilution and so on) but it took maybe 2 years after the acquisition.…

This was cancelled over a year ago - which the articles notes and is old news. It was clear the effort would have needed a very significant push that would have required a large ha…

I left LinkedIn 1.5 years ago. I was there 12 years. I saw the revenue & profitability growth that occurred post acquisition. I am very very confident LinkedIn would be worth north…

Will be a long time before a large number of these are available in clouds though, not until summer at least.