Has anyone used Gemini Code Assist? I'm curious how it compares with Github Copilot and Cursor.
God, it sucks
Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
31–40 of 151 posts
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#32Has anyone used Gemini Code Assist? I'm curious how it compares with Github Copilot and Cursor.
The Aider leaderboards seem like a good practical test of coding usefulness: https://aider.chat/docs/leaderboards/. I haven't tried Cursor personally but I am finding Aider with Sonnet more useful that Github Copilot and its nice to be able to pick any model API. Eventually even a local model may be viable. This new Gemini model does not rank very high unfortunately.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#33Has anyone tried Google's context caching feature? The minimum caching window being 32k tokens seems crazy to me.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#34As someone who actually had to build on Gemini, it was so indefensibly broken that I couldn't believe Google really went to production with it. Model performance changes from day to day and production is completely unstable as Google will randomly decide to tweak things like safety filtering with no notice. It's also just plain buggy, as the agent scaffolding on top of Gemini will randomly fail or break their own internal parsing, generating garbage output for API consumers.
Trying to build an actual product on top of it was an exercise in futility. Docs are flatly wrong, supposed features are vaporware (discovery engine querying, anybody?), and support is nonexistent. The only thing Google came back with was throwing more vendors at us and promising that bug fixes were "coming soon".
With all the funded engagements and credits they've handed out, it's at the point where Google is paying us to use Gemini and it's _still_ not worth the money.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#35This sounds interesting:
"We will continue to offer a suite of safety filters that developers may apply to Google’s models. For the models released today, the filters will not be applied by default so that developers can determine the configuration best suited for their use case."
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#36Earlier quoted context omitted.
I do like the trend. Imagine if Anthropic or someone eventually release a Claude 3.5 but at like a whopping 10x its current speed. Would be incredibly more useful and game changing than a slow o1 model that may or not be x percent smarter.
We might see that with the inference ASICs later this year I guess?
Ooh, what are these ASICs you're talking about? My understanding was that we'll see AMD/Nvidia gpus continue to be pushed and very competitive as well as have new system architectures like cerebras or grok. I haven't heard about new compute platforms framed as ASICs.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#37Google should just offer llama3 405b, maybe slightly fine tuned. Geminis are unusable.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#38As someone who actually had to build on Gemini, it was so indefensibly broken that I couldn't believe Google really went to production with it. Model performance changes from day to day and production is completely unstable as Google will randomly decide to tweak things like safety filtering with no notice. It's also just plain buggy, as the agent scaffolding on top of Gemini will randomly fail or break their own int…
> Docs are flatly wrong
This +999; I couldn't believe how inconsistent and wrong the docs were. Not only that, but once I got something successfully integrated, it worked for a few weeks then the API was changed, so I was back to square one. I gave it a half-hearted try to fix it but ultimately said 'never again'! Their offering would have to be overwhelmingly better than Anthropic and OpenAI for me to consider using Gemini again.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#39Gemini feels like an abusive relationship — every few months, they announce something exciting, and I’m hopeful that this time will be different, that they’ve finally changed for the better, but every time, I’m left regretting having spent any time with them. Their docs are awful, they have multiple unusable SDK's and the API is flaky. For example, I started bumping into "Recitation" errors - ie they issue a flat out…
i think it's unusable if you are trying to use via GCP. Using via ai studio is a decent experience
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#40Earlier quoted context omitted.
Makes sense, as soon as your data leaves your computer, it's safe to assume it's no longer private, no matter what promises a service gives you. You want guaranteed private data that won't be used for anything? Keep it on your own computer.
Not sure why your getting down voted. Anything sent to an cloud hosted LLM is subject to be publicly released or used in training. Setting up a local LLM isn't that hard, although I'd probably air gap anything truly sensitive. I like ollama, but it wouldn't surprise me if it's phoning home.
This is just incorrect. The OpenAI models hosted though Azure are HIPAA-compliant, and Antropic will also sign a BAA.