Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Waiting Time Paradox, Or, Why Is My Bus Always Late?

jakevdp.github.io

Post reply on HN
Print view View original

21–30 of 98 posts

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#21

Post by lordnacho »

twtw wrote: post #13

> a Poisson process is a memoryless process that assumes the probability of an arrival is entirely independent of the time since the previous arrival. In reality, a well-run bus system will have schedules deliberately structured to avoid this kind of behavior: buses don't begin their routes at random times throughout the day, but rather begin their routes on a schedule chosen to best serve the transit-riding public.…

> What is an actual phenomenon that is well modeled by a poisson process?

Time to next Bitcoin block mined. It's 10 minutes, regardless if whether you've waited 1 hour, 10 minutes, or 10 seconds.

Makes sense though, because all the failed hashes are useless, thus no memory.

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#22

Post by QML »

lordnacho wrote: post #21
twtw wrote: post #13

> a Poisson process is a memoryless process that assumes the probability of an arrival is entirely independent of the time since the previous arrival. In reality, a well-run bus system will have schedules deliberately structured to avoid this kind of behavior: buses don't begin their routes at random times throughout the day, but rather begin their routes on a schedule chosen to best serve the transit-riding public.…

> What is an actual phenomenon that is well modeled by a poisson process? Time to next Bitcoin block mined. It's 10 minutes, regardless if whether you've waited 1 hour, 10 minutes, or 10 seconds. Makes sense though, because all the failed hashes are useless, thus no memory.

To add on, there was this article posted months ago on the same topic:

Why Is It Taking 20 Minutes to Mine This Bitcoin Block?

https://news.ycombinator.com/item?id=16469382

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#23

Post by hammock »

mmt wrote: post #9
jchw wrote: post #3

Earlier quoted context omitted.

That would probably make it seem, though, that the buses actually don't arrive (on average) every 10 minutes, since you'd oversample the buses that take longer than 10 minutes.

I'd argue that it's not over sampling at all, but, rather, that the measure of "average bus arrival time" is what's invalid or misleading. After all, the point of the bus arrivals isn't in service of the bus (or driver) but of the passengers. Observed average wait time at each bus stop is a better measure. The even better measure would be average wait time weighted by number of passengers [1]. [1] which is tougher to…

That's a nice idea but ignores all the people sitting in their offices or homes, choosing to go or not go out of their places down to the bus stop.

Better to consider each bus stop as an asset to invest in, the more valuable it is, the more people you can serve.

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#24

Post by QML »

stephengillie wrote: post #5

OneBusAway is surprisingly accurate, at least in my experience. Google Maps has very good transit support too. One reason buses are late is because a bus must travel a circuit. Cars provide linear transportation, so the delay can only happen in the direction of your travel. Since buses run a circuit, they are impacted by delays in the direction opposite of your travel as well. Your bus might be late because the retur…

Why is modeling required? Can’t we just put a sensor on every bus, and just return the empirical expected time it takes for the next bus to drive to your station given the time of day and day of week?

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#25

Post by bostonpete »

herodotus wrote: post #8

Nice article. It reminds me of my year living in London, and taking the bus everyday to Imperial College from West End Lane in West Hampstead. There was a stop on both sides of the road - one for the outbound bus, and one for the inbound (the bus went from central London to a terminus and then returned mostly on the same route). Now we did not use schedules - way too inaccurate at rush hour, and the busses there were…

Your use of "of course" seems to imply that there's some statistical reason that the probability of the next bus being inbound vs outbound wouldn't be equal. Is there? If so, it seems like it must be a different reason than the one in the article. What am I missing...?

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#26

Post by ikken »

This reminds me the bet in the bitcoin community [1]. If on average bitcoin blocks are produced every 10 minutes, and you learn that 5 minutes ago someone found a block, what is the average time you will wait for the next block? It turns out it's 10 minutes, not 5 minutes as you would intuitively think. (it's a memoryless process, so average expected time till block is always the same - 10 minutes - no matter how many blocks were recently found).

In other words, when you're waiting for bitcoin transaction to be confirmed and go to check how long ago the most recent block was produced, in order to estimate how soon the next one will come - you're doing it wrong. Even if previous block was found 9 minutes ago, you're average waiting time for the next block is still 10 minutes.

[1]. https://www.reddit.com/r/btc/comments/7rs8ko/dr_craig_s_wrig...

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#27

Post by stephengillie »

QML wrote: post #24
stephengillie wrote: post #5

OneBusAway is surprisingly accurate, at least in my experience. Google Maps has very good transit support too. One reason buses are late is because a bus must travel a circuit. Cars provide linear transportation, so the delay can only happen in the direction of your travel. Since buses run a circuit, they are impacted by delays in the direction opposite of your travel as well. Your bus might be late because the retur…

Why is modeling required? Can’t we just put a sensor on every bus, and just return the empirical expected time it takes for the next bus to drive to your station given the time of day and day of week?

Every bus does have sensors - it's where the data in the post came from. OneBusAway shows the live location of your bus too.

The issue here is the deviation between empirical expected time and actual arrival time. Unhandled exceptions abound.

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#28

Post by usgroup »

This is a straightforward consequence of modelling an arrival process as a Poisson distribution with a constant rate of arrival lambda...

Go from arrival to cumulative arrivals to time of arrival to recurrence of arrival (next arrival). All are Poisson processes, including the recurrence process, which has a fixed expected value.

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#29

Post by theoh »

bostonpete wrote: post #25
herodotus wrote: post #8

Nice article. It reminds me of my year living in London, and taking the bus everyday to Imperial College from West End Lane in West Hampstead. There was a stop on both sides of the road - one for the outbound bus, and one for the inbound (the bus went from central London to a terminus and then returned mostly on the same route). Now we did not use schedules - way too inaccurate at rush hour, and the busses there were…

Your use of "of course" seems to imply that there's some statistical reason that the probability of the next bus being inbound vs outbound wouldn't be equal. Is there? If so, it seems like it must be a different reason than the one in the article. What am I missing...?

It is impossible to catch more than one inbound bus on any given occasion, whereas any number of outbound buses might pass.

BTW, on a slightly unrelated point, if there's no timetable, but the interval between buses is maintained reliably, the expected waiting time is uniformly distributed over that interval.

If you have to get a second bus, you need to convolve two of those two uniform distributions to find out the distribution of overall journey times. This is a trapezoidal distribution, which is just about analytically manageable.

But a journey with two transfers (3 buses in total) results in a likely overall time distributed according to a uniform distribution convolved with a trapezoidal distribution, which is a very weird non-smooth shape. You can see why people choose to model distributions with Gaussians, which are well-behaved (convolve two Gaussians, get another Gaussian). The Gaussian just lends itself ideally to recursive applications, hence recursive filtering (e.g. Kalman filters).

Re: The Waiting Time Paradox, Or, Why Is My Bus Always Late?

#30

Post by dmurray »

bostonpete wrote: post #25
herodotus wrote: post #8

Nice article. It reminds me of my year living in London, and taking the bus everyday to Imperial College from West End Lane in West Hampstead. There was a stop on both sides of the road - one for the outbound bus, and one for the inbound (the bus went from central London to a terminus and then returned mostly on the same route). Now we did not use schedules - way too inaccurate at rush hour, and the busses there were…

Your use of "of course" seems to imply that there's some statistical reason that the probability of the next bus being inbound vs outbound wouldn't be equal. Is there? If so, it seems like it must be a different reason than the one in the article. What am I missing...?

It's not the next bus, it's the number of buses going the other way, which will on average be greater than 0.5 by the same reasoning given in the article.

It's also possible for the result to be biased because of scheduling. If inbound buses pass every 10 minutes at 16.00, 16.10, 16.20,... and outbound buses at 16.01, 16.11, 16.21, ... you'll usually see an inbound bus first. Though I expect this was not the case here.

Post reply on HN