Avoiding Flat Tires in your Web Application
phase2technology.com
Avoiding Flat Tires in your Web Application
1–10 of 14 posts
Re: Avoiding Flat Tires in your Web Application
#2and a simple "Cache-Control: max-age=30, must-revalidate" http header ...
Re: Avoiding Flat Tires in your Web Application
#3and a simple "Cache-Control: max-age=30, must-revalidate" http header ...
That helps with the same client hitting your website over and over. For a multitude of clients, who's cache is cold, this would not help.
Re: Avoiding Flat Tires in your Web Application
#4and a simple "Cache-Control: max-age=30, must-revalidate" http header ...
That helps with the same client hitting your website over and over. For a multitude of clients, who's cache is cold, this would not help.
I've updated my comment above to say "and" instead of "or" as you're quite correct.
From the timeline of events (surviving initial onslaught, then falling over) I'd assumed that was a large part of the problem, people sitting on a live-updating map.
Re: Avoiding Flat Tires in your Web Application
#5Interesting, and useful info.
I am developing a similar application with a map and live data generated dynamically by my server. The difference with my app, is that the data will be global, not for one city.
The Citi Bike Map appears to load all of the data available regardless of map extent, which can be practical with a small dataset.
How do you deal with caching when the extent of the user's map is constantly changing? My server currently returns data relevant to the map extent only. If I were to get Cron to regularly crunch out one big global cache to send to the browser, I would kill my site.
I am assuming the best option would be to figure out some sort of tiling scheme.
If anyone out there has any ideas about that, I am all ears!
Re: Avoiding Flat Tires in your Web Application
#6and a simple "Cache-Control: max-age=30, must-revalidate" http header ...
That helps with the same client hitting your website over and over. For a multitude of clients, who's cache is cold, this would not help.
The browser is not the only place that can cache HTTP responses - proxy caches from the client's ISP and/or local WAN or corporate intranets should also obey caching rules in the HTTP response headers.
HTTP caching works. The problem with the bike share site in my opinion is instead of using established methods for scaling (like HTTP caching) and other good practices they reinvented the wheel as a Rube Goldberg monstrosity which, big surprise, didn't work.
Remember kids - don't scale until you need to.
Re: Avoiding Flat Tires in your Web Application
#7Interesting, and useful info. I am developing a similar application with a map and live data generated dynamically by my server. The difference with my app, is that the data will be global, not for one city. The Citi Bike Map appears to load all of the data available regardless of map extent, which can be practical with a small dataset. How do you deal with caching when the extent of the user's map is constantly chan…
You should look into vector map tiles! Basically, you create a little JSON snippet for every image that the map server returns, and then overlay it in the client. There's support for JSON tiles in Polymaps (" rel="nofollow">http://polymaps.org/>).
Re: Avoiding Flat Tires in your Web Application
#8The best part of this analysis was the tone - he was polite, non-judgemental, and understanding of the fact that it's tough to launch something new and yes, the developers could have done it better, but no need to pile-on, they're probably kicking themselves enough as is.
Re: Avoiding Flat Tires in your Web Application
#9Interesting, and useful info. I am developing a similar application with a map and live data generated dynamically by my server. The difference with my app, is that the data will be global, not for one city. The Citi Bike Map appears to load all of the data available regardless of map extent, which can be practical with a small dataset. How do you deal with caching when the extent of the user's map is constantly chan…
I wouldn't worry about the caching, as your app is unlikely to get the same news coverage as the citi bikes program. I think the author misdiagnosed the citi bikes problem. My guess is they just had too few apache processes. If there was just one database table, the database's query cache should have been able to handle this load. His suggestions of cron based caching are likely redundant and overkill. Even if that wasn't enough, PHP's APC would be a better solution.
You need a way to do geospatial queries and likely clustering of the results. PostGIS will work well, and there are kmeans clustering plugins, or you can just do grid based clustering.
Re: Avoiding Flat Tires in your Web Application
#10Interesting, and useful info. I am developing a similar application with a map and live data generated dynamically by my server. The difference with my app, is that the data will be global, not for one city. The Citi Bike Map appears to load all of the data available regardless of map extent, which can be practical with a small dataset. How do you deal with caching when the extent of the user's map is constantly chan…
I wouldn't worry about the caching, as your app is unlikely to get the same news coverage as the citi bikes program. I think the author misdiagnosed the citi bikes problem. My guess is they just had too few apache processes. If there was just one database table, the database's query cache should have been able to handle this load. His suggestions of cron based caching are likely redundant and overkill. Even if that w…
Are you saying that PostGIS has a way of handling the marker clustering rather than dealing with clustering in the browser (like I am now doing)?
If so, that would lighten up the data being sent back from the server significantly.