Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autocomplete from Stack Overflow

emilschutte.com

Post reply on HN
Print view View original

101–110 of 151 posts

Re: Autocomplete from Stack Overflow

#101

Post by jandrese »

Tau_Zero wrote: post #69
hbt wrote: post #62

It still astounds me that we haven't solved reusable modules in 2016. Sure, we have libraries, apis, package managers etc. but every time I read a code base, there is always a utility function reinventing the wheel. Someone wrote it because it is still difficult to discover modular code and reuse it easily. It's pretty nuts when you think about it. Imagine mechanical engineers having to recreate the same CAD file bec…

One big reason I've seen people "reinventing" small utilities and modules is to avoid external dependencies, both for licensing and management hassle.

This is a big one. External dependencies are a liability, you need to be getting enough value out of them to offset their cost. For small one-off functions it's often better to just rewrite the thing than to pull in a library and have to worry about versioning or the library being abandoned or the API changing or the package manager on your target system not having that library or only having an old version or one that was compiled weirdly and doesn't have the function you need. There can also be interface friction between your code and the library--different syntactic style, or you have to convert the arguments from your native form into something the library can understand and then convert the result back.

Don't get me wrong, libraries are wonderful and useful and you should be using them where appropriate, but like everything they have tradeoffs. I've seen way too many projects that try to just tie every single library they can find together without having to write any code. I know these projects because they're a nightmare to install and keep working and I end up spending way too much time trying to hack around all of the code rot to get them running again.

Re: Autocomplete from Stack Overflow

#102

Post by chillacy »

conjectures wrote: post #84
ajmurmann wrote: post #53

I had a conversation with a coworker a few days ago where I jokingly suggested building an AI system that you give a failing test suite and it uses stackoverflow answers to make your tests pass. Sounds like we are one step closer to making that happen.

That actually sounds realistic. A well defined test suite is probably a good target for AI. Main issue is no partial wins, which are needed for training, so you'd need to break all functionality down into absolutely minimal units.

Also if you overfit, your code is going to look like:

  int add(int a, int b) {
    if (a == 1 && b == 1) return 2;
    if (a == -1 && b == 0) return -1;
    if (a == 13 && b == 7) return 20;
    return 0;
  }
But at least it passes my unit test suite!

Re: Autocomplete from Stack Overflow

#103

Post by TheOtherHobbes »

ThinkCritically wrote: post #37
jonnycowboy wrote: post #33

Earlier quoted context omitted.

A good start would be for Github to host a package that contains the latest versions of each repo. However even they may not have that right (depends on TOS).

What they really need is a way to rank github code by quality - so that a tool like this pulls in the good code (as opposed to code of lesser quality)

Then you build a helper tool that analyses the differences between good code and bad code.

Then you set it loose on its own source - and you have an AI that takes over the world. (Or at least github, which is more or less the same thing.)

Re: Autocomplete from Stack Overflow

#104

Post by dsp1234 »

hbt wrote: post #62

It still astounds me that we haven't solved reusable modules in 2016. Sure, we have libraries, apis, package managers etc. but every time I read a code base, there is always a utility function reinventing the wheel. Someone wrote it because it is still difficult to discover modular code and reuse it easily. It's pretty nuts when you think about it. Imagine mechanical engineers having to recreate the same CAD file bec…

but every time I read a code base, there is always a utility function reinventing the wheel.

Sometimes the wheel is invented because the existing wheel isn't built for the same purpose (ex: off-road).

ex:

I recently needed to check a buffer to see if it's contents were valid UTF-8 in nodejs. I wanted to do this without having to run it through something like StringDecoder, because it's an intermediate buffer, and I didn't want to have the overhead of actually converting it since it wasn't for my use. So I do 'npm install is-utf8'. Then:

  let isUtf8 = require('is-utf8')
  isUtf8(new Buffer('\u0000')) //false
  isUtf8(new Buffer('\u000B')) //false
  isUtf8(new Buffer('foobar')) //true
So the package is-utf8 is really more like "Is a printable UTF-8 string". Which is certainly a reasonable choice, and a possibly reasonable default, but was not what I needed.

Re: Autocomplete from Stack Overflow

#105

Post by firethief »

chillacy wrote: post #102
conjectures wrote: post #84

Earlier quoted context omitted.

That actually sounds realistic. A well defined test suite is probably a good target for AI. Main issue is no partial wins, which are needed for training, so you'd need to break all functionality down into absolutely minimal units.

Also if you overfit, your code is going to look like: int add(int a, int b) { if (a == 1 && b == 1) return 2; if (a == -1 && b == 0) return -1; if (a == 13 && b == 7) return 20; return 0; } But at least it passes my unit test suite!

Just rerun it with a lower target code complexity. You'll know you set it too low when it starts monkey-patching the test framework to avoid actually doing anything.

Re: Autocomplete from Stack Overflow

#106

Post by dsp1234 »

lmm wrote: post #86
skybrian wrote: post #81

Earlier quoted context omitted.

What makes you think that utility function you saw wasn't copied from somewhere?

Maybe it was, but that's a problem if so - it's not going to benefit from updates, and it will have to be maintained on its own indefinitely.

but that's a problem if so

It's not necessarily a problem. It actually cuts both ways.

The code is not going to benefit from updates, but the code is not also going to be harmed by updates. Think API or behavior changes. Additionally, if the use case for a dependency changes, sometimes the "new" solution doesn't match well.

Ex:

I work with jscript in ASP sometimes. A javascript library changed such that instead of doing iterative processing of nested items (pushing an item on to an array, then looping, popping off the item and processing it, then repeating), they changed to use node's nexttick with some logic like 'we want to move to nested function calls for processing, and doing that iteratively would blow the stack, so we'll just use nexttick so it won't have an increasing stack'. Well, jscript in ASP doesn't have nexttick or any equivalent timer. So while the original code itself worked flawlessly, the use case of the authors moved and we pulled that code in as an external dependency.

That's obviously an extreme case, but I can't really count the number of times that an API change in an NPM package for node has meant modifying code, without a change in functionality.

So yes, you get updates, and sometimes those are going to be security updates and real bug fixes. Other times though that update is going be adding new functionality (and possibly new attack vectors), or dropping support for your use case, etc.

Like many things in our field, it's wisest to look at the risks in all cases, evaluate them for the specific situation at hand, and then choose the appropriate one, rather than cargo-culting one 'best practice'.

Re: Autocomplete from Stack Overflow

#107

Post by creed »

Great idea, nice thinking!

I guess we as programmers should seriously think about the future of our craft.

If, for example, we feel we are basically using the same building blocks over and over again, we should seriously think about organizing libraries and code snippets and questions asking for such snippets in an organized way and provide ways to transpile 1 solution in different languages etc..

We should not accept the current state of our craft as final and rather think about how to improve in general.

If for instance something like StackOverflow has become the Wikipedia of Code then let's think hard about how to make into a full blown tool, with all the features and semantics we need. It was a nice project the way it started and grew but it doesn't have to stay like that forever!

Re: Autocomplete from Stack Overflow

#108

Post by falcolas »

redblacktree wrote: post #54
falcolas wrote: post #45

A great way to increase your liability in an automated fashion! Sure, the code on Stack Overflow is licensed as MIT... but what assurance is there that the code which was posted is original property that the poster owns copyright to? What assurance is there that the posters won't claim patents on the methods being used? The risk is low, but it's certainly not 0. Big companies caution their software developers to not…

Which big companies? I've never heard of such a policy.

Microsoft and Oracle are two that I have direct experience with.

Oracle has even refused to take a 3rd party's code and integrate it into their systems, even when all rights were assigned over.

Re: Autocomplete from Stack Overflow

#109

Post by falcolas »

matt4077 wrote: post #63
falcolas wrote: post #45

A great way to increase your liability in an automated fashion! Sure, the code on Stack Overflow is licensed as MIT... but what assurance is there that the code which was posted is original property that the poster owns copyright to? What assurance is there that the posters won't claim patents on the methods being used? The risk is low, but it's certainly not 0. Big companies caution their software developers to not…

It's fortunately not _that_ easy to get a software patent. It'd be quite difficult to patent something that's the length of an average stackoverflow answer. There's also a lower limit on the length (and originality) for copyright which I doubt many answers reach.

I can fit the concept of mp3 decoding and a sample decoder in a few dozen lines of text and code. This is a patented technology (for another year or two, at least).

Re: Autocomplete from Stack Overflow

#110

Post by lsiebert »

hbt wrote: post #76
incepted wrote: post #75

Earlier quoted context omitted.

But we have this in software. On the JVM, Maven repos are essentially an enormous repository of existing modules that you can pull in your project with a one line addition to your build file. This covers small utility libraries to manipulate strings or dates (like the one you allude to) to entire ecosystems like web containers and everything in-between. I don't understand what part of this doesn't qualify as a "reusa…

npm and maven modules are not easily discoverable - How do you search for a function other than by description or name. Assuming the developer classified properly and you even happen to use the same domain language as the writer. - When multiple modules are returned, similar packages, how do you compare and select efficiently without wasting hours reading the code, evaluating it, checking if it is still maintained an…

A random thought. Linting and static analysis tools for code smells might be correlated with bad code, in which case a tool that let you see that package A has huge monolithic functions and poor test and Package B is modular and simple and good test coverage might be a good way to decide between modules that do the same thing.
Post reply on HN