[deleted]
Devin: AI Software Engineer
541–550 of 604 posts
Re: Devin: AI Software Engineer
#542Earlier quoted context omitted.
I've participated in programming olympiads and worked with people of varying skill and I would say that overall the correlation between competitive programming and software engineering in a business environment is probably like 0.2 or less.
> 0.2 or less I find that questionable. What does "software engineering in a business environment" require that a competent competitive programmer couldn't also learn?
Specification. For any real business, it takes huge effort for a group of people across many domains to consolidate what should be done. That's only the what part.
Not saying competitive programming contest easy or something, but just pointing out that in a contest with timing constraint, the requirement realization phase cannot be fitted in.
Another analogy: martial art vs. military.
Re: Devin: AI Software Engineer
#543Earlier quoted context omitted.
Have you tried Groq? We did a few days testing on replacing gpt4-turbo with it and, while incredibly fast, the results were horrible, even after a lot of specific prompt engineering. So many hallucinations and such. Our products all have to do with strict generation and software quality; it basically has to fill in the blanks but it was incredibly hit or miss. Some results came in within a second so even a few iterat…
Groq is hardware not software... It's like saying the H100 hallucinated.
Well, why not. It's like saying that Windows crashed - while it actually was some driver or app that caused it.
The hardware (or OS) is useless if it doesn't give good results, even if it theoretically could.
Re: Devin: AI Software Engineer
#544Earlier quoted context omitted.
> It's something a clever fourth-grader would write. This level of cope and denial is amazing to witness. The most powerful (multi trillion dollar) companies on the planet are pouring practically infinite resources into developing systems that will ultimately make you redundant. An early version of AGI is staring you in the face while you call it a "fourth-grader". It won't stay in fourth grade forever.
I don't think I'm particularly in denial about the prospects of AI. I think it's going to be hugely disruptive and could possibly put me out of a job. But I'd like to posit a hypothetical counterpoint, just to get you thinking. So far, all of the work on AGI has been the result of brute forcing. We've tried to develop a structural understanding of how the human brain works, and we've failed. So we've fallen back to t…
>So far, all of the work on AGI has been the result of brute forcing. We've tried to develop a structural understanding of how the human brain works, and we've failed. So we've fallen back to torturing circuits into reorienting themselves into compression algorithms for human knowledge. The mechanisms that these tortured circuits used for doing so, the structures they produced in N-dimensional space to embody that knowledge -- we have very little understanding of how these things actually work under the hood.
And this is the way. (Machine) learning theory is in some way a meta-science about how to do science from facts in order to construct theories that effectively explain these facts. What you are asking for will never amounts to a short set of equations. There is not elegant theory of how to perceive numbers and this is why symbolic artificial perception, rule engines, spam detection, RDF ontologies, etc never took off. You're idealizing knowledge as a set of representations without ever reifying how these representations come into existence. We're departing a world of representation toward a world driven by "incarnations": you can't make sense of a how a brain works without the help of another brain, and this is why there is so many things being researched at the intersection of deep learning and neuroscience. I'd even go as far as considering this is in fact how brains work: they can be composed and decomposed monoidically.
In short:
>a structural understanding
There is no such thing
>the structures they produced in N-dimensional space [...] this brute force approach
This is a contradiction. I'm not saying there won't be "structural insights along the way" nor that throwing categories into the machine learning mix won't be useful, but the learning-like aspect that you denote by "brute force" is more fundamental, and in some way above the very processus of science.
Re: Devin: AI Software Engineer
#545Earlier quoted context omitted.
I use ChatGPT every day and it’s excellent at: - replacing StackOverflow and library documentation - library search - converting between formats and languages - explaining existing code/queries - deobfuscating code - explaining concepts (kinda hit or miss) - helping you get unstuck when debugging or looking for solution (‘give me possible reasons for …’) I feel like many of this things require asking the right questi…
> replacing StackOverflow and library documentation I find it horrible at replacing library documentation > I feel like many of this things require asking the right questions, which assumes certain level of experience. But once you reach this level, it’s an extremely valuable assistant. I've been using LLM products since incipience. I use them in my daily work life. It's a bit tiring hearing this 'right questions', '…
I have noticed that ChatGPT tends to give higher quality results the more your question looks like professional technical writing. And on the other end, the more casual or student like your writing style is the lower quality the result.
So you can increase the quality of the response by writing your question in the same kind of style you would see in great documentation. Include lot's of details, be very specific, ask about one thing.
Re: Devin: AI Software Engineer
#546Earlier quoted context omitted.
It's worth pointing out that on their eval set for "issues resolved" they are getting 13.86%. While visually this looks impressive compared to the others, anything that only really works 13.86% of the time, when the verification of the work takes nearly as much time as the work would have anyway, isn't useful. The problem with this entire space is that we have VC hype for work that should ultimately still be being do…
> It's worth pointing out that on their eval set for "issues resolved" they are getting 13.86%. While visually this looks impressive compared to the others, anything that only really works 13.86% of the time, when the verification of the work takes nearly as much time as the work would have anyway, isn't useful. Yeah, I remember speech recognition taking decades to improve, and being more of a novelty - not useful at…
I can see this being useful already (assuming context length is not an issue) as some sort of github service trying to solve github issues throughout the day.
Or for example if you commit todo's in your code, the ai will pick up on them and give you some options later on.
If the failure rate is 14%, just let it try a bunch of times. (half joking here)
The way I see it is at least the project issues are getting some attention, which is arguably better than no attention. If it can just fix simple things, at least you can focus on the complex things and not worry about postponing the low hanging fruits.
Re: Devin: AI Software Engineer
#547Earlier quoted context omitted.
> It's worth pointing out that on their eval set for "issues resolved" they are getting 13.86%. While visually this looks impressive compared to the others, anything that only really works 13.86% of the time, when the verification of the work takes nearly as much time as the work would have anyway, isn't useful. Yeah, I remember speech recognition taking decades to improve, and being more of a novelty - not useful at…
I can see this being useful already (assuming context length is not an issue) as some sort of github service trying to solve github issues throughout the day. Or for example if you commit todo's in your code, the ai will pick up on them and give you some options later on. If the failure rate is 14%, just let it try a bunch of times. (half joking here) The way I see it is at least the project issues are getting some a…
What about Youre Holding It Wrong?
IE: Maybe rather than throwing it find problems, also use it to just build things from scratch - as an example:
The side hustle: make that a product:
Let someone like me who is not a coder have access to Devin for a month with the only goal of building a side hustle that brings a solo person a monthly income.
Then - Sell that so that the millions of people who have a solo idea and just need that "technical co-foumder" can use it to build. and limit it to one devin instance to a person to start...
I dont want it to do a one fell swoop - I'd like to say "build this module...
--
Have a contest where you get What can you build with Devin in TOPIC in 30 mins.
Re: Devin: AI Software Engineer
#548Earlier quoted context omitted.
> 0.2 or less I find that questionable. What does "software engineering in a business environment" require that a competent competitive programmer couldn't also learn?
Specification. For any real business, it takes huge effort for a group of people across many domains to consolidate what should be done. That's only the what part. Not saying competitive programming contest easy or something, but just pointing out that in a contest with timing constraint, the requirement realization phase cannot be fitted in. Another analogy: martial art vs. military.
When I was a kid in the 80s - I convinced my dad he needed a 286 for his construction company - so he could do his books. And he'd need a modem - its the new fax.
(This was so I could play populous, BBS to San Jose and get Grounded for a month for running the phone bill up to 926 for long distance calling into PCLink...
My dad yelled at me for playing video games "WHAT THAT EVER GUNNA DO"
--
years later I left my shift at Intel running the game lab.. to meet my dad for dinner.
He apologized to me for telling me that games and computers would never do anything.
I was touched he remembered.
---
These dumb olympiads will never amount to much, id bet.
Re: Devin: AI Software Engineer
#549I must say, I'm not HUGELY impressed with a website that lets me, unauthenticated, upload files of an arbitrary size. Just posted a 500mb dmg file to their server. If anyone is practicing for their B1 Dutch exam, feel free to use this link to get the practice paper. https://usacognition--serve-s3-files.modal.run/attachments/4...
Looks like they deleted it and restricted file uploads :/
The first rule of goofy-bug-found on demo is you dont talk about goofy-bug-found on demo
Re: Devin: AI Software Engineer
#550Earlier quoted context omitted.
There is not a limited amount of software engineering that can be done. There's only the amount of software engineering that it is _economical_ to do at any given point. If AI makes software development cheaper and more efficient, people will just use apply it to more use cases. It's never been the case that making programming cheaper has lead to fewer programmers -- quite the opposite. This change is roughly analogo…
The problem is that AI has the potential to turn a professionalized industry into a self-help one. Like, if I can pay $30 a month to have an AI make me several new apps a day, why would I need an app store? It's like fast fashion for programs. Low quality but dirt cheap.
But thats OK!
THis is exactly what I want.
I dont want to describe some enterprise architure with DR and blah.
No - I want the little utils that I just want to appear like little magic fairy dust...
Little apps. This is how a new eco system grows.