Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Varnish 4.0.2 released

varnish-cache.org

Post reply on HN
Print view View original

31–40 of 56 posts

Re: Varnish 4.0.2 released

#31

Post by bkeroack »

latch wrote: post #22

My experience with Varnish was when we tried to use it for something that had relatively complex caching rules. The config/VCL became spaghetti: attaching values to req and restarting the flow. This appeared to be the normal way to write VCL. We ended up writing our own system, with our own high-concurrent LRU cache, that was more tightly coupled with our application servers (and thus able to figure out what the cach…

Agreed. Varnish is great if you have an existing web application that needs a quick-and-dirty caching layer built in front of it.

Varnish gives you enough configurability to do just about anything, but the end result will be a spaghetti mess of VCL, inline C and possibly custom varnish modules. The more sustainable route is to build your own caching front end, however I think it's a bit unrealistic to assume that all startups have the engineering talent (or time) in-house to implement that sort of thing.

Re: Varnish 4.0.2 released

#32

Post by bkeroack »

phkamp wrote: post #18
imaginenore wrote: post #15

Varnish left a sour taste in my mouth. We have used it on one of our high-traffic sites, and had some truly bizarre hard-to-reproduce problems with it. The major bug it had was that it would work normally for hours, but then it would randomly let the flood of requests through basically killing our servers. It happened over and over and over again. We had multiple talented sysadmins look at it, and none of them could…

That sounds like a "hit-for-pass" scenario: Something from your backend told Varnish "Don't cache this" and varnish stopped doing so.

This. The thing to remember is that you always have to set beresp.ttl in vcl_fetch in a hit-for-pass situation. Varnish caches the decision to hit-for-pass (or lookup or whatever), so if you do a hit-for-pass and your TTL is 1 hour, Varnish will hit-for-pass that cache key for the next hour without running your VCL logic again.

Re: Varnish 4.0.2 released

#33

Post by imaginenore »

phkamp wrote: post #18
imaginenore wrote: post #15

Varnish left a sour taste in my mouth. We have used it on one of our high-traffic sites, and had some truly bizarre hard-to-reproduce problems with it. The major bug it had was that it would work normally for hours, but then it would randomly let the flood of requests through basically killing our servers. It happened over and over and over again. We had multiple talented sysadmins look at it, and none of them could…

That sounds like a "hit-for-pass" scenario: Something from your backend told Varnish "Don't cache this" and varnish stopped doing so.

Except we didn't have that. This was 99.9% content pages dynamically generated and nearly static. They all had roughly the same headers that haven't changed really, and they were not even aware of Varnish.

Response headers is the first thing we checked, there are only a few of them that affect Varnish:

https://www.varnish-software.com/static/book/HTTP.html

Re: Varnish 4.0.2 released

#34

Post by ruben_varnish »

imaginenore wrote: post #15

Varnish left a sour taste in my mouth. We have used it on one of our high-traffic sites, and had some truly bizarre hard-to-reproduce problems with it. The major bug it had was that it would work normally for hours, but then it would randomly let the flood of requests through basically killing our servers. It happened over and over and over again. We had multiple talented sysadmins look at it, and none of them could…

Next time call me and I will make sure that the best minds doing cache invalidation in the industry have a look and fix your issue.

[Varnish Software sales hat on]

Re: Varnish 4.0.2 released

#35

Post by spacemanmatt »

bkeroack wrote: post #31
latch wrote: post #22

My experience with Varnish was when we tried to use it for something that had relatively complex caching rules. The config/VCL became spaghetti: attaching values to req and restarting the flow. This appeared to be the normal way to write VCL. We ended up writing our own system, with our own high-concurrent LRU cache, that was more tightly coupled with our application servers (and thus able to figure out what the cach…

Agreed. Varnish is great if you have an existing web application that needs a quick-and-dirty caching layer built in front of it. Varnish gives you enough configurability to do just about anything, but the end result will be a spaghetti mess of VCL, inline C and possibly custom varnish modules. The more sustainable route is to build your own caching front end, however I think it's a bit unrealistic to assume that all…

Wow, this (and parent-post) really colors my understanding about how far one can/should go with Varnish before building something more tailored. Thanks.

Re: Varnish 4.0.2 released

#36

Post by eugeneionesco »

brokentone wrote: post #30

Varnish and caching in general can be a complete mind-freak the first few times you experience it, however after working with it nearly daily for the last 3 years, I've come to love it. It has allowed me to scale a number of WordPress websites far beyond where they had business being (34mm UV/180mm PV per month on a handful of servers). Super excited to see them continuing to build in great features.

>34mm UV/180mm PV per month on a handful of servers

Can you give more details about the hardware setup?

Re: Varnish 4.0.2 released

#37

Post by ochoseis »

Two questions: (1) Anyone use Varnish in front of Rails? Is it better to just set up some memcached and/or redis and let the app handle it? (2) To deal with session-specific stuff, how common is it to render some common plain html views, then fill in the particulars with JS/Ajax? If you need to support no-JS (heaven forbid) do people use iframes?

Re: Varnish 4.0.2 released

#38

Post by seanp2k2 »

ochoseis wrote: post #37

Two questions: (1) Anyone use Varnish in front of Rails? Is it better to just set up some memcached and/or redis and let the app handle it? (2) To deal with session-specific stuff, how common is it to render some common plain html views, then fill in the particulars with JS/Ajax? If you need to support no-JS (heaven forbid) do people use iframes?

Check out ESI blocks: https://www.varnish-cache.org/trac/wiki/ESIfeatures

Varnish will very likely be an order of magnitude or two faster than sending requests to your app. You can get going with a simple config in a few minutes and without breaking anything (just have varnish listen on a different port and use :80 or your app server as the back-end), so I'd strongly suggest playing with it if you're curious. It's pretty impressively fast software, and VCL gives you an awesome abstraction for controlling how the cache handles different routes.

Re: Varnish 4.0.2 released

#39

Post by seanp2k2 »

imaginenore wrote: post #33
phkamp wrote: post #18

Earlier quoted context omitted.

That sounds like a "hit-for-pass" scenario: Something from your backend told Varnish "Don't cache this" and varnish stopped doing so.

Except we didn't have that. This was 99.9% content pages dynamically generated and nearly static. They all had roughly the same headers that haven't changed really, and they were not even aware of Varnish. Response headers is the first thing we checked, there are only a few of them that affect Varnish: https://www.varnish-software.com/static/book/HTTP.html

Are you sure it wasn't an issue with cache evictions?

Re: Varnish 4.0.2 released

#40

Post by kawsper »

Shish2k wrote: post #24
krat0sprakhar wrote: post #3

We've been long vacillating on using Varnish to power our E-commerce store but from what we've read, Varnish is not a good solution in cases where cookies are part of most of web traffic. Is this true?

In addition to other answers - it depends what the cookies are for -- if your users only get a session cookie after logging in (before that they only have front-end cookies for things like google analytics / remembering which widget on the site is currently active / etc) then you can ignore front-end cookies, thus successfully caching for all not-logged-in users: if(req.http.Cookie) { # ignore front-end cookies (pref…

This is the way to do it.

Another way of doing it is to exclude all cookies, and only care about session-cookies.

Post reply on HN