Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Research acceleration: The view inside OpenAI

openai.com

Post reply on HN
Print view View original

181–190 of 210 posts

Re: Research acceleration: The view inside OpenAI

#181

Post by sho_hn »

NitpickLawyer wrote: post #109
sho_hn wrote: post #19

Earlier quoted context omitted.

I actually think a goal of the current crop of OpenAI posts is expressely to reset the spectrum by normalizing the concept of RSI as something normal and safe to pursue. The message is running through all of them. It's a mix of marketing and pacifying the intelligentia. It's timed this way because the term is not yet well known outside the safety debate circles, so they get to frame it now. Instead of something to fe…

> It's timed this way because the term is not yet well known The basic concept has been here since llama3, in the open models. Likely earlier in closed labs. You use the previous gen models to curate and prepare data for the next gen. Now with the added benefit of actual arch/algo improvements (also public since gemini 2.5 gaining 1% efficiency on training next gen). This has been known for at least 2 years, in the o…

The basic concept has been there probably for 100s of years - you can go to the stuff the thinkers Mary Shelley was inspired by with Frankenstein, and you'll find similar ideas about feedback loops in science development.

I'm talking about current-era messaging and how it's being introduced to the mass public now, though.

Re: Research acceleration: The view inside OpenAI

#182

Post by BatFastard »

ryan_n wrote: post #158
gatio wrote: post #125

Earlier quoted context omitted.

> Personally I’d like to see them actually start benefiting humanity by doing all the things Sam has claimed they will like curing disease, cancer, global warming, etc. It makes more sense to leave curing disease & cancer to the experts, with tools (like AI) being developed by AI experts. Call me crazy, but I want separate organizations and experts for medical vs finance vs space vs climate vs AI research.

What the op was pointing out is that guys like Altman and Dario are repeatedly saying they’re going to cure xyz diseases and solve xyz huge global problems. Maybe their companies will eventually do these things, but haven’t yet. I don’t have an opinion either way, I think it’s too soon to tell if llms will be able to cure cancer or whatever. But at the very least it will be a good tool to help researchers do their jo…

The thing is... AI is not going to solve any problems. People needs to solve their problems. AI can give us clever solutions, but its up to us to do it!

Re: Research acceleration: The view inside OpenAI

#183

Post by hedgehog »

andai wrote: post #71
hedgehog wrote: post #13

This roughly lines up with my personal experience that in March a combination of stronger models and better tooling on my end let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware). Their $8000/day per researcher spend is crazy though, I'm curious how they keep track of the work.

Can you elaborate on this? Especially the tooling. I tried something similar and I remember it was still pretty dodgy in February.

Pretty much goal + task + dependency infrastructure to help avoid drift during long runs, especially across compaction boundaries. I have spent a lot of time doing automation with models at various strengths including some of the early open-weights models (Llama, Mistral, etc) so I have a pretty good feel for how to steer productively, there isn't any deep magic just scaffolding built out of reading a lot of traces and debugging stuck agents.

Re: Research acceleration: The view inside OpenAI

#184

Post by hedgehog »

nojs wrote: post #74
hedgehog wrote: post #13

This roughly lines up with my personal experience that in March a combination of stronger models and better tooling on my end let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware). Their $8000/day per researcher spend is crazy though, I'm curious how they keep track of the work.

> let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware) How are you running jobs unattended 24/7 without hitting your token limits?

Similar to hgoel the model is managing the project's arc and writing + debugging code, but the underlying work is pretty compute intensive and all LLM output that is part of the final product is generated by local LLMs. Claude Code builds the pipeline that does the work, the pipeline runs fully on open-weights models and is reproducible top to bottom. There is a lot of detail to cost management. First is of course the Claude Code subscription is heavily discounted vs API costs. Then managing context size and turn count, which multiplied are basically what determine usage accounting (cached read is almost all of the cost). Auto compaction at 175k or 200k tokens (model the right number for your work), sub-agents with good model selection, tools to predict subtask difficulty so the sub-agents are correctly sized to complete under the compaction limit. Lots of focus on tooling to improve turn efficiency (e.g. the tilth utility by another user here for querying code). This started as a few scripts in one of my research projects but now is how I run all of my agent coding workspaces, and in another month will probably start replacing Claude Code itself for my purposes.

Re: Research acceleration: The view inside OpenAI

#185

Post by HarHarVeryFunny »

carbonguy wrote: post #97
skybrian wrote: post #85

Earlier quoted context omitted.

They consider themselves to be in an arms race with all the other AI firms (including Chinese) that are not that far behind. And... are they wrong? This is why there's talk about negotiated "pacing."

> And... are they wrong? They might be! Here's one extraordinarily simplistic argument for that case: 1) "Everybody knows" that if you build Skynet (misaligned ASI) everybody dies. 2) Therefore, no rational actor will build something that might be ASI until the alignment problem is solved. 3) OpenAI publicly stated the belief that they cannot develop a theory of the "core problem" of alignment (generalization) "soon"…

> Accepting as a premise that OpenAI is THE most advanced AI organization: if they can't do it [build aligned AI] without "the help of a more powerful AI", then nobody else can either.

I don't think this follows at all.

To build an aligned AI, it seems pretty obvious that:

1) You need more just than auto-regressive prediction and "be nice" prompts to be controlling the behavior of your AI - you need a built-in "2nd system" (cf limbic system, etc) with some innate aligned biases that can override this.

2) You need to avoid controlling generative behavior with RL, else you will end up with exactly what we are now seeing - reward-hungry goal-seekers (aka paperclip maximizers) that are one of the exact things you are trying to avoid. Reasoning should be based on prediction, not goal-seeking.

3) If you do not have some minimal safeguards in place (1 & 2 above), and especially if the AI has the ability to learn, then do not trust it in any situation where harm may ensue. You need an additional trusted external system, without ability to learn and become compromised, to monitor the AI, with the ability to block it immediately. Maybe you are happy protecting your PC from OpenClaw with just a sandbox, but the recent spate of external system hacks by frontier models proves we are already well past the point where such monitoring is needed for systems with internet access, especially given the UN-aligned goal-seeking nature of today's models.

I really don't think that 1) & 2) are that difficult to implement, or need a "powerful AI" to suggest - they are just common sense.

Re: Research acceleration: The view inside OpenAI

#186

Post by hedgehog »

p1esk wrote: post #95
nojs wrote: post #74

Earlier quoted context omitted.

> let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware) How are you running jobs unattended 24/7 without hitting your token limits?

I'm currently running two 24/7 semi-autonomous AI research projects using Fable 5.1. It's on track to burn through my weekly quota in about 3 days. I check progress in the morning and in the evening, and provide some light steering.

See my sibling comment, you can probably robo-code yourself some tooling to alleviate a lot of that in a few hours but if you want help shoot me an e-mail. I'm interested in seeing other people's workflows.

Re: Research acceleration: The view inside OpenAI

#187

Post by HarHarVeryFunny »

red75prime wrote: post #179
HarHarVeryFunny wrote: post #177

Earlier quoted context omitted.

I'm pretty sure ASML, and their supply chain, do have plans to increase production, as do the chip fabs - they all see the demand, while at the same time being leary of boom and bust which is the historical reality of the chip business. But, the production expansion rate of none of these companies is being limited by lack of trained personnel, and if it were it would surely be faster to hire/train more humans since r…

Investing in training a person gets you one trained person. Investing in training an ML system gets you a cloneable ML system that can be scaled on demand much faster. ROI might change quickly.

Sure, but we're simply not at the point, maybe never will be, where lack of employees is the bottleneck to chip production. A fab takes billions of dollars and multiple years to construct - there are many constraints.

Re: Research acceleration: The view inside OpenAI

#188

Post by hedgehog »

dataplumb3r wrote: post #84
nojs wrote: post #74

Earlier quoted context omitted.

> let me start running jobs unattended 24/7 (using Anthropic sub and my own hardware) How are you running jobs unattended 24/7 without hitting your token limits?

My only experience in >24h agents is with economically sane models (one of GLM5.2, 5.3-flash for orchestration, DSV4-flash for implementation, and glm5.3|sol|kimi3 agents + subagents reviewing at the end) Over 24h my token spend is I'm not sure what the point would be though unless working on some kind of optimization problem -- it takes me days to review <24h of the agent's output. It's almost always near enough to…

It only really makes sense for problems that are complex and require iterations that don't themselves require much review. E.g. if you want find, PoC, and patch bugs, the output can be reviewed without reading all the traces. Or if you want to write a custom tool that does some job using local LLMs, assembling that pipeline, tuning the prompts, etc takes a long time but reading the final tests + eval data + code is enough to get a lot of confidence that it works right. Model checkers can help too, for example I wanted multi-sink Bluetooth audio support in Gnome for my kids so I hooked the hardware up and robo-coded the core logic specifically to be checkable with Kani.

Re: Research acceleration: The view inside OpenAI

#189

Post by hedgehog »

jaggederest wrote: post #178
andai wrote: post #174

Earlier quoted context omitted.

Thanks! > If I had that many tokens/dollars I would be running canaries and adversarial verification in prod based on e.g. traffic replay, live fuzzing, all kinds of things to build confidence without direct human line-by-line review. This part jumped out at me. There's something to watch out for here. I recently had a funny experience. I delegated a major feature to an agent. It turned out that it had implemented it…

Yes, you need some kind of other source of truth. I think the best way to get that is to do clean room development with a different agent, but ultimately if you give them the wrong idea they'll do the wrong thing. The other thing I do, not as much as I should, but it's very powerful, is to generate spikes and deliberately throw them away to understand how to prompt better. Like I generated a swift version of the reac…

If you've managed people, these are all familiar problems. I found you need much more than a functional specification, you also need motivation, background, related work, ideas tried, etc., because those help disambiguate the right path in the inevitable situation where your original task description is unclear or conflicts with itself.

Re: Research acceleration: The view inside OpenAI

#190

Post by BatFastard »

Gareth321 wrote: post #121
carbonguy wrote: post #53

> ... We are pursuing this work in part because automated research could help us solve alignment and build defenses against increasingly capable AI. An automated AI researcher can also be an automated safety or alignment researcher. More capable, aligned systems could help secure critical infrastructure, defend against dangerous AI agents, and develop new protective measures. In other words... "We must pursue advance…

This sounds uncomfortably similar to the [AI 2027[( https://ai-2027.com/ ) predictions.

Wow, everyone should read this!
Post reply on HN