Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Continuous Diffusion Language Models (CDLM's)

sander.ai

Post reply on HN
Print view View original

21–30 of 55 posts

Re: Continuous Diffusion Language Models (CDLM's)

#21

Post by pests »

ramon156 wrote: post #8
NickNaraghi wrote: post #3

I wonder if we’ll get something like CDLMs for automated harness engineering, sort of piloting the LLM underneath.

how do tools like hermes do this? does it just review sessions and rewrite markdown files? also haven't read too deep into the deepseek agent harness but the math in there was really cool. it sounded promising, at least.

> does it just review sessions and rewrite markdown files

Yes, same with openclaw etc. Some might have plugins to integrate with graph or vector databases besides only markdown files.

Re: Continuous Diffusion Language Models (CDLM's)

#22

Post by slashdave »

echelon wrote: post #11
janalsncm wrote: post #7

> [in 2020/2021] the dominance of autoregression was not as well-established as it is today: GPT-3 had turned some heads, but the ‘ChatGPT moment’ wouldn’t come until late 2022 I disagree with this. Decoders were absolutely dominant in 2020 for chat. GPT2 was considered too dangerous to release, and I remember scrambling to get on the GPT3 waitlist. It worked. (The only exception I will make is encoder-decoder models…

> GPT2 was considered too dangerous to release This is how ridiculous this industry is. Regulation-seeking panic over nothing. Drama in search of a moat. Everything is "too dangerous". GPT2 is going to invent a time machine and break crypto and genetically engineer super rabies. They sell knives, guns, combustible materials, and multi-ton heavy machinery in stores. That's what's actually dangerous.

Well, no. Without controls, a language model can drive sensitive individuals to violence or suicide. The idea of releasing a frontier model without RL is frightening based on what we have learned.

Re: Continuous Diffusion Language Models (CDLM's)

#24

Post by idiotsecant »

alightsoul wrote: post #19
janalsncm wrote: post #12

Earlier quoted context omitted.

My impression at the time was they were perhaps overly cautious but this was a bunch of researchers who wanted to self-regulate. Anthropic didn’t exist, deepmind was also much less product-focused and relatively cautious. Chinese models weren’t really a factor either. Government regulation was really not in the picture either in 2020 or 2021 tbh. The government was still trying to beat a pandemic. Some of the Biden a…

why are chinese models a consideration at all?

Parent post is saying there was no community of peer models, just some people figuring it out as they went and with lots of slack to go slow if they wanted

Re: Continuous Diffusion Language Models (CDLM's)

#25

Post by latentsea »

echelon wrote: post #11
janalsncm wrote: post #7

> [in 2020/2021] the dominance of autoregression was not as well-established as it is today: GPT-3 had turned some heads, but the ‘ChatGPT moment’ wouldn’t come until late 2022 I disagree with this. Decoders were absolutely dominant in 2020 for chat. GPT2 was considered too dangerous to release, and I remember scrambling to get on the GPT3 waitlist. It worked. (The only exception I will make is encoder-decoder models…

> GPT2 was considered too dangerous to release This is how ridiculous this industry is. Regulation-seeking panic over nothing. Drama in search of a moat. Everything is "too dangerous". GPT2 is going to invent a time machine and break crypto and genetically engineer super rabies. They sell knives, guns, combustible materials, and multi-ton heavy machinery in stores. That's what's actually dangerous.

What are your thoughts on the HuggingFace incident?

Re: Continuous Diffusion Language Models (CDLM's)

#26

Post by anon373839 »

slashdave wrote: post #22
echelon wrote: post #11

Earlier quoted context omitted.

> GPT2 was considered too dangerous to release This is how ridiculous this industry is. Regulation-seeking panic over nothing. Drama in search of a moat. Everything is "too dangerous". GPT2 is going to invent a time machine and break crypto and genetically engineer super rabies. They sell knives, guns, combustible materials, and multi-ton heavy machinery in stores. That's what's actually dangerous.

Well, no. Without controls, a language model can drive sensitive individuals to violence or suicide. The idea of releasing a frontier model without RL is frightening based on what we have learned.

> The idea of releasing a frontier model without RL is frightening

In case you were not aware, strong base models (no post training at all) have been available for quite some time now. Including ones that eclipse “scary” frontier models from even a year ago.

Re: Continuous Diffusion Language Models (CDLM's)

#27

Post by latentsea »

ramon156 wrote: post #8
NickNaraghi wrote: post #3

I wonder if we’ll get something like CDLMs for automated harness engineering, sort of piloting the LLM underneath.

how do tools like hermes do this? does it just review sessions and rewrite markdown files? also haven't read too deep into the deepseek agent harness but the math in there was really cool. it sounded promising, at least.

There's a cool research project https://github.com/exoharness/exo that is designed specifically as a self-modifying harness, so architecturally it separates things in a way that makes it a lot harder for the agent to break itself when modifying itself. It looks pretty neat. Saw the author do an interview on a podcast explaining it.

Re: Continuous Diffusion Language Models (CDLM's)

#28

Post by [deleted] »

alightsoul wrote: post #19
janalsncm wrote: post #12

Earlier quoted context omitted.

My impression at the time was they were perhaps overly cautious but this was a bunch of researchers who wanted to self-regulate. Anthropic didn’t exist, deepmind was also much less product-focused and relatively cautious. Chinese models weren’t really a factor either. Government regulation was really not in the picture either in 2020 or 2021 tbh. The government was still trying to beat a pandemic. Some of the Biden a…

why are chinese models a consideration at all?

[deleted]

Re: Continuous Diffusion Language Models (CDLM's)

#29

Post by CamperBob2 »

latentsea wrote: post #25
echelon wrote: post #11

Earlier quoted context omitted.

> GPT2 was considered too dangerous to release This is how ridiculous this industry is. Regulation-seeking panic over nothing. Drama in search of a moat. Everything is "too dangerous". GPT2 is going to invent a time machine and break crypto and genetically engineer super rabies. They sell knives, guns, combustible materials, and multi-ton heavy machinery in stores. That's what's actually dangerous.

What are your thoughts on the HuggingFace incident?

"Drama in search of a moat" is hard to beat.

Re: Continuous Diffusion Language Models (CDLM's)

#30

Post by SV_BubbleTime »

slashdave wrote: post #22
echelon wrote: post #11

Earlier quoted context omitted.

> GPT2 was considered too dangerous to release This is how ridiculous this industry is. Regulation-seeking panic over nothing. Drama in search of a moat. Everything is "too dangerous". GPT2 is going to invent a time machine and break crypto and genetically engineer super rabies. They sell knives, guns, combustible materials, and multi-ton heavy machinery in stores. That's what's actually dangerous.

Well, no. Without controls, a language model can drive sensitive individuals to violence or suicide. The idea of releasing a frontier model without RL is frightening based on what we have learned.

So…

Words out of a magic box on a computer can’t make you kill yourself. Perhaps the people that would use that as encouragement are already mentally ill enough that it really doesn’t matter what the trigger is?

On a more callus but fully serious note, evolution starts out physical, that the species that don’t eat and breed as well, die. What happens when you remove that? When life is so safe that you basically wont stave, get eaten, catch a disease, don’t really need to compete all that hard for scarce resources, etc? Do you think evolution just stops? Or perhaps does a social or mental evolution become the predominant differentiator for successful reproduction over time?

Post reply on HN