GitHub Copilot available for JetBrains and Neovim

The example i usually give [1] in javascript is say you have function

    (x,y) => x + y

Orgs targeting code coverage write a test for 1,2 => 3 and get 100% coverage and then stop as there is no incentive to go further. They don't write tests for say

   (1,null)
   (null,null)
   ('x',1) 
   (NaN, Infinity) 

and so on.. these additional tests will improve coverage of scenarios and code coverage will not move.

I have seen projects where a test will have sequence of steps which trigger the code but the assertion is effectively true === true, or they will replicate the same function in the test instead of generating proper mock data or myriad different absurd testing approaches. This comes from twin pressure of showing coverage and having tests pass.

Coverage is also a challenge in code which uses AI/ML libraries or that use third party services. These really need statistical testing with large volume of diverse well maintained samples and results need statistical analysis for error rates not different from how manufacturing does it, I don't see that often. For code using face detection for example a single face getting detected or not is hardly an adequate test.

Finally, it is easier to improve coverage by testing simpler code than improve coverage/ refactor a function which has say 10 nested branches, so it is not uncommon to see 90% coverage and 10% of the most used / most error prone code poorly or not tested at all.

There are some methods to address these like mutation testing, do retros for failure of tests to capture production bugs, it is not easy to measure and coverage driven orgs will not see their metrics moving by doing these.

Well written test suites will also have good coverage, but not necessarily other way around. Developers who care and understand what they are doing and why they are doing it, will use coverage as only the first step to see where there are gaps in their tests.

Tests are also code that need peer reviewed and maintained, if the tests depend on implementation and constantly break or contains improper mocks or assert poorly. A lot of not well written tests is hindrance to development than aid it.

[1] Yes, most of these are not applicable in a strongly typed language, but it is far easier as a illustration .

I’ve been using this for weeks and it blooooows my mind. It comes up with crazy recommendations, just yesterday I wrote this big ass logic to do something, then I wanted to move that code to a function so I wrote the function name and I kid you not copilot suggested a one-liner that worked… the thing is so useful and I’m not writing simple code (writing cryptographic code). And when it’s not doing that at the very least it provides auto-completion to lists where a counter has to increase and things like that. It’s just baffling, I don’t think I’m that directly impacted by AI, or at least this is the first time where I’m like “wow, AI really is changing the world RIGHT NOW”. Half-joking: can’t wait for it to just write my code.

Also if this feature would be paid tomorrow, I think I would pay for it. It’s really noticeable when I don’t have copilot enabled now.

Oh and, autocompletion doesn’t work with markdown files because of markdown plugins I think? But this is another level of insanity: when I’m writing english it figures a lot of the sentences I want to write. Makes me question if I’m just a deterministic individual with no choice.

Copilot is crazy. The other day, I was writing a Python function that would call a Wikipedia API. I pulled from the internet an example of a GET request, and pasted it as a comment in my code.

  # sample call: https://en.wikipedia.org/w/api.php?action=query&format=json&list=geosearch&gscoord=37.7891838%7C-122.4033522&gsradius=10000&gslimit=100

Then I defined a variable,

  base_url = "https://en.wikipedia.org/w/api.php?"

Then, like magic, Copilot suggested all the remaining keys that would go in the query params. It even knew which params were to be kept as-is, and which ones would come from my previous code:

  action = "query"  # action=query
  format = "json"  # or xml
  lat = str(latitude.value)  # 37.7891838
  lon = str(longitude.value)  # -122.4033522
  gscoord = lat + "%7C" + lon
  ...
  api_path = base_url + "action=" + action + "&format=" + format + ... + "&gscoord=" + gscoord

As a guy who gets easily distracted while programming, Copilot saves me a lot of time and keeps me engaged with my work. I can only imagine what it'll look like 10 years from now.

How could they support these two random IDEs before Visual Studio proper?

I would say the reverse, I’m getting so many hits that I’m mindblown. And when it missed, I can generally still use that and fix the suggestion, as it’s faster.