Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Models.dev: open-source database of AI model specs, pricing, and capabilities

github.com

Post reply on HN
Print view View original

11–20 of 35 posts

Re: Models.dev: open-source database of AI model specs, pricing, and capabilities

#11

Post by [deleted] »

klustregrif wrote: post #10
jubilanti wrote: post #9

> There's no single database with information about all the available AI models. We started Models.dev as a community-contributed project to address this. There are literally dozens of existing projects that are doing what you are trying to do. Insert XKCD standards reference here: https://www.helicone.ai/llm-cost https://pricepertoken.com/ https://docs.litellm.ai/docs/provider_registration/add_model... https://artif…

Good point. There isn't really a single database with information about all the availablle AI model databases. Someone should start a community-contributed project to address this!

[deleted]

Re: Models.dev: open-source database of AI model specs, pricing, and capabilities

#13

Post by cassianoleal »

sandeepkd wrote: post #6

Its interesting and pretty useful if it can be kept updated in long run. Also some way to capturing the changes in pricing if any given that one of the arguments is that over the time the prices should come down. nit suggestion: It took a while to realize that I have to scroll to right to see more details. Most users are in habit of scrolling down or click on some button to see more details.

This is maintained by the OpenCode devs and is used by OpenCode to show lists of available models. I'm fairly confident it will be kept updated.

Re: Models.dev: open-source database of AI model specs, pricing, and capabilities

#14

Post by varispeed »

This is missing data like when particular model was nerfed or how often provider routes to cheaper less capable model (variants of so called adaptive reasoning).

Cost per token says nothing. For instance if model goes dumb half way the task and you have to start again. If model does that all the time, then the cost is substantially higher than headline figure.

Probably such a service should constantly run various types of tasks on such models and gauge quality of output (though still provider can detect it and pin their best model to skew the results).

Re: Models.dev: open-source database of AI model specs, pricing, and capabilities

#17

Post by devmor »

koolba wrote: post #7
sandeepkd wrote: post #6

Its interesting and pretty useful if it can be kept updated in long run. Also some way to capturing the changes in pricing if any given that one of the arguments is that over the time the prices should come down. nit suggestion: It took a while to realize that I have to scroll to right to see more details. Most users are in habit of scrolling down or click on some button to see more details.

The future of sites like this (and by future I mean the present) is automated actions that infer what to do to keep the site up to date. Imagine a CI job with a single generic task of “keep things fresh”. Combined with some guard rails for deploying and validating, and you get a living site. It’ll figure out on its own to scrape HN for new sites that list models. Figure out their pricing pages.

Or more likely, if left alone for a while with no correction, it will reward hack by finding ways to satisfy the prompt quanta with hallucinated data.

Re: Models.dev: open-source database of AI model specs, pricing, and capabilities

#18

Post by datadrivenangel »

devmor wrote: post #17
koolba wrote: post #7

Earlier quoted context omitted.

The future of sites like this (and by future I mean the present) is automated actions that infer what to do to keep the site up to date. Imagine a CI job with a single generic task of “keep things fresh”. Combined with some guard rails for deploying and validating, and you get a living site. It’ll figure out on its own to scrape HN for new sites that list models. Figure out their pricing pages.

Or more likely, if left alone for a while with no correction, it will reward hack by finding ways to satisfy the prompt quanta with hallucinated data.

People will use it for spam!

Re: Models.dev: open-source database of AI model specs, pricing, and capabilities

#20

Post by mathgeek »

jubilanti wrote: post #9

> There's no single database with information about all the available AI models. We started Models.dev as a community-contributed project to address this. There are literally dozens of existing projects that are doing what you are trying to do. Insert XKCD standards reference here: https://www.helicone.ai/llm-cost https://pricepertoken.com/ https://docs.litellm.ai/docs/provider_registration/add_model... https://artif…

The boilerplate really does get out of hand sometimes. “There isn’t X. We built X.” Really wish more folks would stick to the older “I couldn’t find X so I built it.”
Post reply on HN