Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What we learned in 6 months of working on an AI Developer

blog.pythagora.ai

Post reply on HN
Print view View original

31–40 of 51 posts

Re: What we learned in 6 months of working on an AI Developer

#31

Post by __loam »

bugglebeetle wrote: post #18
berkes wrote: post #11

Earlier quoted context omitted.

> Pretty soon GPT-4 will not be the best in the field. The next generation will perform much better. What makes you believe that progress is linear, or at least a line forever going up? I keep seeing people predicting rapidly improving AI, based on how rapid it improved over the last x months. But why is that not an outlier? How do we know we haven't hit a ceiling and stagnating? Isn't progress typically very bumpy a…

I would say that Microsoft/OpenAI’s attacks on open source, whether it be through “AGI safety” BS as a front for regulating their way to a monopoly, or attempting Embrace, Exentend, Extinguish on companies like Mistral, and Cold War-style fear mongering about China, are the greatest near-term risks to linear progress. And it’s worth noting on that latter point that China is not similarly constrained and so could end…

The greatest near term risk to progress here is cost but nobody wants to talk about it.

Re: What we learned in 6 months of working on an AI Developer

#32

Post by teaearlgraycold »

berkes wrote: post #11
Lerc wrote: post #7

Even though I don't think GPT-4 is up to the task, it does seem like now is the right time to be working on these things. Pretty soon GPT-4 will not be the best in the field. The next generation will perform much better. Possibly the most frustrating thing I find about GPT-4 is how close it gets with it's wrong answers. It's easy to dismiss a lesser answer when it responds with a laughably out-of-band idea. GPT-4 oft…

> Pretty soon GPT-4 will not be the best in the field. The next generation will perform much better. What makes you believe that progress is linear, or at least a line forever going up? I keep seeing people predicting rapidly improving AI, based on how rapid it improved over the last x months. But why is that not an outlier? How do we know we haven't hit a ceiling and stagnating? Isn't progress typically very bumpy a…

Technological innovation is not truly an exponential. It is instead a series of logistics. If we do not have further breakthroughs then AI technology will plateau. I do think we'll have those breakthroughs but it's impossible to say when they will happen.

Re: What we learned in 6 months of working on an AI Developer

#33

Post by [deleted] »

ctoth wrote: post #13

One of the things they seem to have figured out is the requirement to at least model a sort of actor-critic architecture with their agents. It helps quite a bit. They seem to badmouth Aider a tad (not cool) but I do wonder how a full-stack of this + Aider might work? There needs to also be some sort of good test generator involved. All that said, any time someone actually demonstrates progress on the automated Softwa…

[deleted]

Re: What we learned in 6 months of working on an AI Developer

#34

Post by bugglebeetle »

__loam wrote: post #31
bugglebeetle wrote: post #18

Earlier quoted context omitted.

I would say that Microsoft/OpenAI’s attacks on open source, whether it be through “AGI safety” BS as a front for regulating their way to a monopoly, or attempting Embrace, Exentend, Extinguish on companies like Mistral, and Cold War-style fear mongering about China, are the greatest near-term risks to linear progress. And it’s worth noting on that latter point that China is not similarly constrained and so could end…

The greatest near term risk to progress here is cost but nobody wants to talk about it.

Architecture improvements and stuff like the 1-bit networks that have been all over HN recently are going to drastically reduce training costs over time and could easily result in very powerful local models. That’s why there is a rush to regulate and create monopolies. Microsoft and OpenAI know they have no real moat relative to the current pace of R&D.

Re: What we learned in 6 months of working on an AI Developer

#35

Post by wokwokwok »

Hm.

It’s easy to look at https://github.com/Pythagora-io/gpt-pilot-db-analysis-tool/b... and go… so, this new tool means you took two days to write this?

long stare

Why did you bother?

…but, this both hits the nail on the head and misses the point at the same time.

On the one hand, this is foundational tech, prototyping on a new way of doing things. It’s not going to be faster than doing it yourself at first. It won’t run locally at first.

On the other hand, we already know that GPT4 level models can do trivial tasks.

Over and over and over, people claim coding tools can massively improve productivity, and then try to demo that by building a trivial system.

…but building a trivial systems is not the problem that needs solving.

The problem that needs solving is building large complex systems with dynamically adjusting requirements.

The examples and blog post seem to miss this even as an idea.

While I applaud, in general, efforts to explore this space, tackling the easy problems seems like it doesn’t significantly advance the state of play.

Here are some concrete things that would be more valuable, but are significantly technically harder:

- Use tests. Make it write tests. Make humans write tests. Do not accept generated code that fails the tests.

- Focus on refactoring; it’s a known issue that models struggle to refactor code. Breaking your existing code base into tiny files isn’t the answer.

- Focus on documenting the behaviour of existing code and incrementally migrating to new behaviour.

- Bad developers write new code instead of reading the existing code and using existing functionality and utilities. AI generators are notoriously rubbish at this, and will almost always generate a function rather than use an existing one.

Refining and understanding existing code is significantly more valuable than generating code “from scratch”; so much so that I would argue that without the ability to refine existing code, such tools will forever remain in the “scaffold generator” category of “useful but ultimately no better than the current status quo”.

The tool as shown, is I believe broadly speaking interesting, but the approach described in the blog (upfront decisions about everything) is a dead end.

Re: What we learned in 6 months of working on an AI Developer

#36

Post by threeseed »

viksit wrote: post #27
threeseed wrote: post #26

Earlier quoted context omitted.

> China is not similarly constrained and so could end up outcompeting the U.S. With what technology ? US has long term export controls on China and as they have demonstrated with Russia recently once you have secondary sanctions in place everyone falls into line. So it's pretty likely they will be effective.

China actually has very advanced technology in literally every area of the sciences now. They just don’t bother talking about it in english I recommend you follow some of the specialist chinese AI substacks to see what’s happening over there. Esp around chip building

Not really buying the idea that China secretly has equivalents for Nvidia, ASML, TSMC etc.

SIMC right now relies entirely on existing US/EU hardware for their chip building.

New versions of which are no longer available to them.

Re: What we learned in 6 months of working on an AI Developer

#37

Post by sdsd »

viksit wrote: post #27
threeseed wrote: post #26

Earlier quoted context omitted.

> China is not similarly constrained and so could end up outcompeting the U.S. With what technology ? US has long term export controls on China and as they have demonstrated with Russia recently once you have secondary sanctions in place everyone falls into line. So it's pretty likely they will be effective.

China actually has very advanced technology in literally every area of the sciences now. They just don’t bother talking about it in english I recommend you follow some of the specialist chinese AI substacks to see what’s happening over there. Esp around chip building

>I recommend you follow some of the specialist chinese AI substacks to see what’s happening over there.

Do you recommend any in particular? I'm not familiar enough with the Chinese AI scene to know who to check out

Re: What we learned in 6 months of working on an AI Developer

#38

Post by c0balt »

samus wrote: post #17
amelius wrote: post #6

Until I see an AI sysadmin that can help with basic configure/make problems, I don't have high hopes for an AI developer.

That should be quite easy compared to software development, which is much more open-ended since the requirement are usually more nebulous, potentially contradictory, and at times simply wrong.

> That should be quite easy

SysAdmin stuff is quite easy in terms of complexity to some sw stuff. The problems, similar to traditional engineering, tend to come from the rather high cost of failure.

To expand further, it's easy to setup a system but hard to setup one that's reliable and/ or resilient. It's hard to maintain systems that are not documented and/ or wrongly documented (outdated, inaccurate). It's even harder to always make sure everything's consistent and you don't lose/ damage data.

Re: What we learned in 6 months of working on an AI Developer

#39

Post by throwaway2990 »

threeseed wrote: post #26
bugglebeetle wrote: post #18

Earlier quoted context omitted.

I would say that Microsoft/OpenAI’s attacks on open source, whether it be through “AGI safety” BS as a front for regulating their way to a monopoly, or attempting Embrace, Exentend, Extinguish on companies like Mistral, and Cold War-style fear mongering about China, are the greatest near-term risks to linear progress. And it’s worth noting on that latter point that China is not similarly constrained and so could end…

> China is not similarly constrained and so could end up outcompeting the U.S. With what technology ? US has long term export controls on China and as they have demonstrated with Russia recently once you have secondary sanctions in place everyone falls into line. So it's pretty likely they will be effective.

The export controls simply don’t work. Those chips still make their way into China. All those export controls do is slow it down a bit.

But China can outfit itself with more hardware even if it’s not as fast as the latest iteration and still speed past the U.S. while the U.S. and the EU argue about AI being racist or not.

Re: What we learned in 6 months of working on an AI Developer

#40

Post by jaggederest »

__loam wrote: post #28
withinboredom wrote: post #12

Earlier quoted context omitted.

Oh man. When it’s so close but wrong it’s amazing for creative endeavors! For technical ones, it is quite a bad thing. It’s like being a Star Wars fan but the AI just wants to talk about Star Trek. I think this is why the non-tech people see AI as so amazing. For anything human and non-technical, the “almost but not quite” nature is a good thing. I was using an AI to help me debug a weird thing (mainly summarizing lo…

I think it's equally shit at creative work too, that's just harder to dismiss as "wrong". It's still wrong, it's just harder to see.

Rapidly expose the issue by asking it to write something funny. Can't tell a joke, even when prompted for really elementary stuff like knock knock jokes or chicken crossing the road, let alone anything sophisticated.
Post reply on HN