Viewing profile — typpo

No need to hack it, we'll add proper support for this.

>> Terra has competitive performance to GPT‑5.5 [while being 2x cheaper]… > To me that means “it’s an inferior product but marketing dictates we try and hide that.” I interpret thi…

Lately my company has been doing a lot of complex accounting and reporting in spreadsheets. Overall was surprised by how well both GPT and Claude handled some of these extremely te…

Nice work! This is like a much better version of Ancient Earth[0], which I made ~10 years ago using GPlates[1]. I like your approach of rendering the map itself from data, which ma…

This is so fun and creative. Congrats on launching!

Thanks to Meta for their work on safety, particularly Llama Guard. Llama Guard 3 adds defamation, elections, and code interpreter abuse as detection categories. Having run many red…

If anyone is interested in evaling Gemma locally, this can be done pretty easily using ollama[0] and promptfoo[1] with the following config: prompts: - 'Answer this coding problem …

Hi HN, I built this open-source LLM red teaming tool based on my experience scaling LLMs at a big co to millions of users... and seeing all the bad things people did. How it works:…

Care to explain why you think so?

The problem in this case is not that it was trained on bad data. The AI summaries are just that - summaries - and there are bad results that it faithfully summarizes. This is an at…

The amount of negativity in these comments is astounding. Congrats to the teams at Google on what they have built, and hoping for more competition and progress in this space.

Paul's benchmarks are excellent and they're the first thing I look for to get a sense of a new model performance :) For those looking to create their own benchmarks, promptfoo[0] i…

Great idea and congrats on shipping the project! I'm curious if you noticed certain models worked better for summarizing and converting to steps. For example, in my projects I've f…

Public benchmarks are broadly indicative, but devs really should run custom benchmarks on their own use cases. Replicate created a Llama 3 API [0] very quickly. This can be used to…

If anyone wants to eval this locally versus codellama, it's pretty easy with Ollama[0] and Promptfoo[1]: prompts: - "Solve in Python: {{ask}}" providers: - ollama:chat:codellama:7b…

I'm working on https://quickchart.io/ , a web API for generating chart images. I've expanded it to a WYSIWYG chart editor at https://quickchart.io/chart-maker/ , which lets you cre…

Congrats on the launch! I've been interested in automatic testset generation because I find that the chore of writing tests is one of the reasons people shy away from evals. Recent…