31–40 of 51 posts
Love what you’re doing. Are you currently open to interns? Would love to connect with you and chat more about using high quality data to help people better train and evaluate their ai agents!
Curious to see how this works out. The flight booking example is interesting because it’s one of the last purchase powers I’d want to hand over to an AI. If it gets a major travel detail wrong, purchases a business class ticket on accident, etc. and I need to adjust the booking by calling the airline, then I’m way less happy than I was if I just bought the ticket myself. Not to mention what happens when Google flight…
Does this simulation really required? There's another YC startup, they're processing PDFs I believe. They didn't train their systems on any simulation.
Edited to reword and add more context.
Curious to see how this works out. The flight booking example is interesting because it’s one of the last purchase powers I’d want to hand over to an AI. If it gets a major travel detail wrong, purchases a business class ticket on accident, etc. and I need to adjust the booking by calling the airline, then I’m way less happy than I was if I just bought the ticket myself. Not to mention what happens when Google flight…
Airlines will love it too. How long until an AI company gets paid to prefer a certain company
AI vibe coding tools already prefer some solutions over others, probably because of training data distribution/post training preferences. This is leading to massive revenue differences and growth compared to companies that have not optimized to be AI agent preferred/in their training data distribution.
I imagine something similar will happen over time, where companies who are in the training data distribution get used by agents more, while others who neglect this get slowly phased out because systems don't know how to use them (out of distribution).
Have you looked at agents from OpenAI and perplexity? Sure, they aren't perfect, but at the same time, they aren't far from near ready. Does this simulation really required? There's another YC startup, they're processing PDFs I believe. They didn't train their systems on any simulation. Edited to reword and add more context.
That being said, there are still a lot of use cases its not good at, and also looking at long trajectory tasks, enterprise work tasks, etc. I imagine those are all still very nascent.
I think we are still very early on computer use, being "production ready" requires probably close to 95%+ accuracy on most tasks and we're not there yet for most use cases.
Very cool - is it possible to simulate this on a live production site (i.e. instead of Halluminate Flights, just test the agent live on Expedia)? Even though you don't have access to the backend json, presumably you could verify the right values were entered in the frontend/UI?
more importantly though are use cases that depend on the data. the data on real google flights/expedia is constantly changing, so it's impossible to build datasets based ground truth, e.g. the answer for a task like "Find the cheapest round-trip flight option from Bologna (BLQ) to Dushanbe (DYU) if I leave on 2026-05-05 and come back on 2026-05-15. Return the total price and the flight numbers for all flights." isn't stable. on our site, we control the data, so that answer is stable (deterministically random). so controlling the whole clone rather than running on the prod site unlocks richer and more repeatable tasks/testing.
lastly, our site runs the exact same locally as deployed, it has zero internet dependencies. so it can be run offline directly on the cluster with no issue for network latency/failures
Curious to see how this works out. The flight booking example is interesting because it’s one of the last purchase powers I’d want to hand over to an AI. If it gets a major travel detail wrong, purchases a business class ticket on accident, etc. and I need to adjust the booking by calling the airline, then I’m way less happy than I was if I just bought the ticket myself. Not to mention what happens when Google flight…
Why are flight bookings the go to example always? For most people, booking a flight happens infrequently, is a non-trivial expense (to your point), and is not that burdensome to do yourself.
I don't know about you, but it takes me hours to book a flight if it's for my family, because I'm usually booking a flight, a car, and a hotel, and I have to constantly min-max the costs between hotels on certain days, flights on certain days, and cars on certain days.
If it's not burdensome for you, then you're either taking very simple trips or you're so rich that you don't care.
Earlier quoted context omitted.
Why are flight bookings the go to example always? For most people, booking a flight happens infrequently, is a non-trivial expense (to your point), and is not that burdensome to do yourself.
> and is not that burdensome to do yourself. I don't know about you, but it takes me hours to book a flight if it's for my family, because I'm usually booking a flight, a car, and a hotel, and I have to constantly min-max the costs between hotels on certain days, flights on certain days, and cars on certain days. If it's not burdensome for you, then you're either taking very simple trips or you're so rich that you do…
I agree it's a burdensome chore!
Just wondering - your hotel stay can't be less than the days between your flight. For car, one can manage to cut down with Uber/public transport, but still turns out to be expensive than a rental car.