81–90 of 127 posts
Im surprised they didn’t try out Gemini. It’s a lot better than Google translate and in my experience this is one use case where Gemini outshines even other frontier models like ChatGPT4
The results have felt much more like natural language to me.
I wrote a book on using LLMs in applications 14 months ago, I am sort of a fan, within reason. I don’t like all the mind-space taken up on comparisons between LLMs on standard test suites because I personally think most instruction tuned models are specifically tuned for the standard tests.
I like to see which models people are choosing to use for their projects, and of course, tools like Ollama make it easy to try many models and get a least a subjective feel for what they can do.
For model comparison I have my own standard little tests that are my own and private, and I can be sure models are not tuned specifically for.
I do the same for commercial LLM APIs and products surfacing models. For example, I have a few short tests I run in Bard to test integration with Google Workspace data, something I am interested in, and it is interesting to track at least subjectively the slow improvement.
Is Google Translate considered "the benchmark to beat"? At least between German and English (both not exactly "fringe languages"), Google Translate quality is still very hit and miss.
But ChatGPT blows it out of the water if you have a specific context and don't need something exactly translated (e.g. formal letter/e-mail writing).
Earlier quoted context omitted.
Orders of magnitude more energy necessary though.
The GPT 3.5 API is cheaper than Google Translate. And in our testing (over a year ago) was better for translation. So I assume in that case the energy use is less?
Earlier quoted context omitted.
The GPT 3.5 API is cheaper than Google Translate. And in our testing (over a year ago) was better for translation. So I assume in that case the energy use is less?
Isn't OpenAI still operating at a loss? You can't infer much from an API bring cheaper if it's being subsidised.
Given they tripped and fell over a money printing machine and then chose to lower their API prices, it would be pretty surprising (but not impossible) if their API prices are currently subsidised.
DeepL already beat Google Translate years ago.
DeepL has its moments but it also has many many failure cases. It particularly is very bad with long text blocks, it commonly completely misses sections of text or repeats the same block of text multiple times.
Yeah this is old news. When i was in China last year google translate seemed to literally never work (looks of confusion trying to do basic interactions in stores). GPT-4 worked perfectly every time, I think google translate might be another soft abandoned project from google
As a sibling mentioned it's probably something about cost, but given the narrow domain and the performance of smaller models on translation tasks, I'm surprised they're still doing the same old thing in 2024....
Earlier quoted context omitted.
Where is the hallucination? It seems in line with the others.
You think this: "Swarming like a swarm of bees. He was carried among the people, hanging from the handle. No matter how good you think about the situation you're in, it's disgusting. Where are you now?" is a comparable translation to this: "No matter how you couch it, riding the subway feels disgusting: you dangle like ripe fruit from a hanging vine, squeezed in among humans swarming like bees." Or this ?: "Being cra…
Could be better, but it communicates the key concepts and emotional tone?
I still find it amazing how LLM translation capabilities are an almost accidental feature. Yet they still managed to leapfrog decades of research and billions of investment dollars in traditional machine translation.
And still, there are people saying that LLMs are useless. I find that even more amazing.