Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
1–10 of 22 posts
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#2It's really interesting to see where all of this is going. I guess a large part of the best practices behind clearly naming things for human interpretation has allowed for training and evaluation of things by ML models too.
Meta, but the speaker sounds eerily close to Mark Zuckerberg.
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#3It's really interesting to see where all of this is going. I guess a large part of the best practices behind clearly naming things for human interpretation has allowed for training and evaluation of things by ML models too. Meta, but the speaker sounds eerily close to Mark Zuckerberg.
Companies will fire 70% boilerplate coders in following years
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#4It's really interesting to see where all of this is going. I guess a large part of the best practices behind clearly naming things for human interpretation has allowed for training and evaluation of things by ML models too. Meta, but the speaker sounds eerily close to Mark Zuckerberg.
Companies will fire 70% boilerplate coders in following years
Or just write 3x more boilerplate. I don't think we are saturated with software yet.
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#5Why would anybody trust this after they faked the last Gemini demo?
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#6Is this where unit tests will be very useful, where you ask it to fix all bugs found, and make sure it passes all unit tests. This is where all the github's public repos will get really interesting forks.
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#7It's really interesting to see where all of this is going. I guess a large part of the best practices behind clearly naming things for human interpretation has allowed for training and evaluation of things by ML models too. Meta, but the speaker sounds eerily close to Mark Zuckerberg.
Companies will fire 70% boilerplate coders in following years
Na we’ll all be moved to perpetual on-call, every day an endless fire drill as hundreds of services are launched on top of the crumbling crash looping burning landscape of millions of services launched last quarter, a Mad Max world of endless adrenaline and New Relic AI-enhanced alerts.
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#8I'm pretty excited about the increased context length (e.g. in my other comment here[0]), but I'm kind of disappointed by the examples here.
The codebase is 100k lines, but the tasks it gave it seemed to be focused on just hundreds of examples. The examples are probably largely independent, so it doesn't seem like this is really flexing anything a relatively simple RAG approach with a much smaller context window couldn't handle. The prompts said "the demo that ...", so it's a matter of identifying the demo in question and looking at just that code, which is a much smaller necessary context. There was the "use the GUI approach from other examples" task, which kind of gets there, but that's kind of another distinct little bit of code.
In other words, while the codebase has lots of lines, the actual inference across them seemed to use relatively few of them, and identifying the relevant lines didn't seem that hard to me based on the tasks given. That means it could be done with some retrieval and a much smaller context window.
From the title, I thought it would be loading 100k into the context and then asking some deeper questions like "find the bug" that spans several function calls or something like that. Something that wouldn't be trivial to accomplish with current techniques.
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#9I want to see something where we have a big piece of code, and a big standards document it purports to implement, and the system can answer questions like "is this part of the spec implemented? Where it is implemented? What does this piece of code mean (w.r.t. the spec)? If I implemented this part of the spec, where would the changes go?"
Re: Problem solving across 100,633 lines of code – Gemini 1.5 Pro Demo [video]
#10Earlier quoted context omitted.
Companies will fire 70% boilerplate coders in following years
Na we’ll all be moved to perpetual on-call, every day an endless fire drill as hundreds of services are launched on top of the crumbling crash looping burning landscape of millions of services launched last quarter, a Mad Max world of endless adrenaline and New Relic AI-enhanced alerts.
but some companies which build reliable old school software will win the market?..