Just wondering if the guys can automate the scraping part, seems unlikely this is usually information that needs to be handpicked or "moderated" by one or more humans.
[deleted]
Scott's Cheap Flights: Growing a small side project into a booming business
41–50 of 244 posts
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#42So, Scott originally accumulated all those miles working for ThinkProgress? How does that work? You buy the flights and expense them back to the company, pocketing the miles?
Yes, it's completely 100% standard practice as other replies already mentioned. What's funny is that the tone of your question suggests he's doing something fraudulent. If you were the company auditor or a government prosecutor, and you didn't know about the standard practice or simply wanted to be a jerk, you could absolutely make a case for fraud. If you were fired or prosecuted for keeping the miles, what would be your counterargument? Everyone else was doing it? That doesn't work for speeding tickets. This is an example of how arbitrary laws can be.
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#43I hate to be the one to ask, but how much moat does this actually have? Couldn't Kayak literally just add a subscribe or alert me later input and have their massive infrastructure automate this very thing?
Some deals require specific starting and end destinations to work properly with a specific carrier. Others can require a stopover in a certain city before continuing on.
The really good deals will even be site specific. All this can be done via good expertise. With scraping and this type of logic, you'd get blocked quickly from the major sites.
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#44Awesome writeup. It's great to see companies that can make this work without taking any funding. I had no idea there were so many people behind this.
I have nothing to add except that I've been a very happy paid customer for several months now. These guys run a fantastic service that has been worth every penny. Yes, some travel companies that have extensive infrastructure could probably do what is being done here, but they don't.
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#45Ouch, their site triggers one of my pet peeves about travel sites, listing prices without clarity whether the deal is one-way or round trip. NYC to Paris: $260 Normal Roundtrip Price: $900 So did people who got that deal save $380? Or did they save $640? Which is it? As feedback to Scott and Brian, this actually makes me hesitate to sign up for premium, because I don't know what level of discount we're talking about…
All of their listings are roundtrip.
I have personally booked sub or around-$400 return trips to Europe from the US using this service.
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#46Ouch, their site triggers one of my pet peeves about travel sites, listing prices without clarity whether the deal is one-way or round trip. NYC to Paris: $260 Normal Roundtrip Price: $900 So did people who got that deal save $380? Or did they save $640? Which is it? As feedback to Scott and Brian, this actually makes me hesitate to sign up for premium, because I don't know what level of discount we're talking about…
Wow, thanks for the feedback!
Almost every single deal we send out is roundtrip. This is actually why we put "normal roundtrip price" instead of just "normal price" to help clarify this.
Maybe instead we should do "Roundtrip NYC to Paris: $260" and then put "Normal Price: $900" to help clear this up.
Thoughts?
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#47Ouch, their site triggers one of my pet peeves about travel sites, listing prices without clarity whether the deal is one-way or round trip. NYC to Paris: $260 Normal Roundtrip Price: $900 So did people who got that deal save $380? Or did they save $640? Which is it? As feedback to Scott and Brian, this actually makes me hesitate to sign up for premium, because I don't know what level of discount we're talking about…
Wow, thanks for the feedback! Almost every single deal we send out is roundtrip. This is actually why we put "normal roundtrip price" instead of just "normal price" to help clarify this. Maybe instead we should do "Roundtrip NYC to Paris: $260" and then put "Normal Price: $900" to help clear this up. Thoughts?
Doesn't sound like you lose anything if you include roundtrip
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#48Just wondering if the guys can automate the scraping part, seems unlikely this is usually information that needs to be handpicked or "moderated" by one or more humans.
Speculation: it's possible that the source sites are actively hostile to scraping and intentionally mess with the layout regularly, or may even have ruled out any kind of automation in their ToS - so if you don't want to get sued or blacklisted, you can do no kind of (detectable) scraping.
Can anyone recommend scraping adapters (businesses or tech) that are robust to this sort of thing? I'm talking about something higher level than, say, Beautiful Soup -- something you can configure to point at an endpoint, essentially request a sql row subscription from it, and not have to mind it too much.
Both the traversal/retrieval and data-interpretation parts seem to have interesting aspects when you consider current website design. Some websites make themselves hard even for humans to read (consider why safari reader mode exists).
This seems like a potentially valuable service, in the sense of being a schlep. I wonder how many places have home-grown scraping efforts as part of their business and how annoying it is for them to maintain.
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#49[deleted]
Re: Scott's Cheap Flights: Growing a small side project into a booming business
#50I just listened to the podcast for Scott's Cheap Flights.
Fantastic job Courtland. Your podcast is one of my favorites in this space.