11–20 of 77 posts
The cost per question seems super high. I can't even think of an API where a single call would cost $1cent. You better have a good pricing model to follow up on this.
Also, wondering if anyone has found research on the inverse of this approach to the problem, i.e., instead of training the model to understand the data, you improve the data to be more understandable to a model? This seems more promising when you are looking at enterprise use cases without much training data. Spider seems like quite a simple dataset compared to the ones I encounter on the job, and LLMs struggle even with those.
The cost per question seems super high. I can't even think of an API where a single call would cost $1cent. You better have a good pricing model to follow up on this.
Really? I sure can. I can even imagine a single API request that triggers a job that costs $100,000. These APIs exist. They're not for individuals.
The cost per question seems super high. I can't even think of an API where a single call would cost $1cent. You better have a good pricing model to follow up on this.
Google Speech-to-Text is one I've dealt with recently; costs over 1¢ to transcribe 1 minute of audio. Almost 8¢/minute for the medical version.
Earlier quoted context omitted.
Google Speech-to-Text is one I've dealt with recently; costs over 1¢ to transcribe 1 minute of audio. Almost 8¢/minute for the medical version.
1¢ per 1minute is fair as it would take the best translator much more time than that. 1¢ per sql query seems less fair to me, but we could argue it would also cost more to ask a dev.
Are you planning on submitting this model to be evaluated against the Spider holdout set? Also, wondering if anyone has found research on the inverse of this approach to the problem, i.e., instead of training the model to understand the data, you improve the data to be more understandable to a model? This seems more promising when you are looking at enterprise use cases without much training data. Spider seems like q…
Are you planning on submitting this model to be evaluated against the Spider holdout set? Also, wondering if anyone has found research on the inverse of this approach to the problem, i.e., instead of training the model to understand the data, you improve the data to be more understandable to a model? This seems more promising when you are looking at enterprise use cases without much training data. Spider seems like q…
Since the issue is often the context, plugging in data dictionaries (and passing those to the LLM) can help
do you not use a vector db and embeddings search to get the table structure, etc ?