Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Online Shop Till You Drop With Your Friends On Plurchase

techcrunch.com

Post reply on HN
Print view View original

31–40 of 52 posts

Re: Online Shop Till You Drop With Your Friends On Plurchase

#31

Post by catch23 »

eserorg wrote: post #29

Plurchase is just a reverse HTTP proxy server looking for an application. It's not too hard to do something similar in perl. 1) start off by downloading a copy of CGIProxy: http://www.jmarshall.com/tools/cgiproxy/ 2) Setup an apache server running CGIProxy 3) Hack the interface box at the top of the screen to include: facebook integration, a meebo chat box, etc... The problem is that this sort of thing tends to _real…

Most reverse proxies do the easy stuff, but all the hard stuff is still on the client side javascript/flash code. It'll probably take you a few hours to hack up one of the existing proxies out there and get "almost" what plurchase has, but you'll soon see that menus won't render in amazon and links just won't work because the link menu system is completely dynamically constructed so most proxies won't take you this far. We have an html rewriter written in javascript on the client side to solve some of those issues. We also have a javascript rewriter written in javascript to proxify the javascript code. If you right-click view-source on one of the proxified frames, you'll see that we go to great lengths to make things work under the cover.

Also, you might notice that you can't ignore all images & static assets, some of the image requests actually modify cookie information so you may need some different level of proxying there -- same goes for css/javascript assets. You'd be surprised what websites do under the cover. Even with all of our technology, there are still some sites that don't function properly and may require some tweaking on our proxy engine. I tested kayak.com yesterday on our proxy and it almost works but there are still some weird issues going on that I'll have to debug further. I mentioned most of these details in a blog post -- had this been the web 1.0 days, proxying might actually be easy, but we wouldn't have the speed of modern javascript interpreters.

I agree with you on some of the ToS stuff that we'd have to potentially work with merchants on. And since we're white-labeling based on domains, we can remove them if they don't want us there, but proxying shopping sites is unlike other sites -- they want customers to buy stuff. Anything that causes a customer to convert better is in their best interest. Large merchants were actually interested in this, but they didn't want partnerships until they saw traction.

Re: Online Shop Till You Drop With Your Friends On Plurchase

#32

Post by eserorg »

catch23 wrote: post #31
eserorg wrote: post #29

Plurchase is just a reverse HTTP proxy server looking for an application. It's not too hard to do something similar in perl. 1) start off by downloading a copy of CGIProxy: http://www.jmarshall.com/tools/cgiproxy/ 2) Setup an apache server running CGIProxy 3) Hack the interface box at the top of the screen to include: facebook integration, a meebo chat box, etc... The problem is that this sort of thing tends to _real…

Most reverse proxies do the easy stuff, but all the hard stuff is still on the client side javascript/flash code. It'll probably take you a few hours to hack up one of the existing proxies out there and get "almost" what plurchase has, but you'll soon see that menus won't render in amazon and links just won't work because the link menu system is completely dynamically constructed so most proxies won't take you this f…

CGIProxy has full javascript support.

It also handles cookies and can dynamically determine whether or not it should proxify images and CSS assets.

CGIProxy also handles flash.

In fact, CGIProxy can fully proxify an SSL-encrypted GMail session out-of-the-box.

CGIProxy is _very_ mature technology. Take a look at the source code -- it handles 100's of special-cases.

Some of the special-cases that CGIProxy handles are truly bizarre. Reading through the CGIProxy source code is actually pretty fascinating.

This brings up an interesting point. If you're interviewing a software engineering candidate, ask them what their favorite language is. Next, ask them to write a simple HTTP server in that language. Finally, ask them to write a reverse HTTP proxy that works with the server they just wrote. This is a surprisingly effective way of separating the wheat from the chaff. Most people with CS degrees are completely incapable of doing this.

Re: Online Shop Till You Drop With Your Friends On Plurchase

#33

Post by immad »

eserorg wrote: post #32
catch23 wrote: post #31

Earlier quoted context omitted.

Most reverse proxies do the easy stuff, but all the hard stuff is still on the client side javascript/flash code. It'll probably take you a few hours to hack up one of the existing proxies out there and get "almost" what plurchase has, but you'll soon see that menus won't render in amazon and links just won't work because the link menu system is completely dynamically constructed so most proxies won't take you this f…

CGIProxy has full javascript support. It also handles cookies and can dynamically determine whether or not it should proxify images and CSS assets. CGIProxy also handles flash. In fact, CGIProxy can fully proxify an SSL-encrypted GMail session out-of-the-box. CGIProxy is _very_ mature technology. Take a look at the source code -- it handles 100's of special-cases. Some of the special-cases that CGIProxy handles are t…

It rewrites the flash and javascript so that they call your reverse proxy instead of the original domain?

Re: Online Shop Till You Drop With Your Friends On Plurchase

#34

Post by catch23 »

eserorg wrote: post #32
catch23 wrote: post #31

Earlier quoted context omitted.

Most reverse proxies do the easy stuff, but all the hard stuff is still on the client side javascript/flash code. It'll probably take you a few hours to hack up one of the existing proxies out there and get "almost" what plurchase has, but you'll soon see that menus won't render in amazon and links just won't work because the link menu system is completely dynamically constructed so most proxies won't take you this f…

CGIProxy has full javascript support. It also handles cookies and can dynamically determine whether or not it should proxify images and CSS assets. CGIProxy also handles flash. In fact, CGIProxy can fully proxify an SSL-encrypted GMail session out-of-the-box. CGIProxy is _very_ mature technology. Take a look at the source code -- it handles 100's of special-cases. Some of the special-cases that CGIProxy handles are t…

I downloaded to take a look at the source, they do things that are similar to us actually, and they do have lots of great special case handling that we might use for reference later. One crazy thing that we do that isn't done here is the storage of cookies, we actually temporarily store cookies on the server side so that we can do fancy things with it -- sending around the shopping cart, transfer of shopping state etc.

However, if you've tried using it, you'll notice that it isn't perfect either... they show slashdot.org as an example, but lots of things are broken on it. Most of the javascript stuff doesn't work. Slashdot was a javascript light site a year ago, but now they use tons of fancy javascript like most sites. Their cookie support also seems to be broken -- not sure why. Although I will admit that James did quite a bit of amazing work in just 12k lines of perl code. The perl code is pretty decent too.

It's easy to get a proxy that works on 95% of the code, all the hard stuff is in that last 5% -- that's what we spent most of the time on :-)

Re: Online Shop Till You Drop With Your Friends On Plurchase

#35

Post by catch23 »

slice wrote: post #28

I noticed that they are using CDN from day one. What's the rational behind it? How much complexity the CDN integration adds to the code and procedures? Is the number of images they load an important factor here? I think they are loading only few images of their own. (I'm not criticizing, just asking).

Actually, I was just bored. I meant to take it out but forgot about it... I was trying to get a high score in YSlow ;-).

If you look into the CSS, it still references the non-CDN images and Safari is actually somewhat stupid -- it loads both sets of images even if the CDN images override it. The homepage actually loads slower with the CDN images (but I get a higher score on YSlow :-P )

Re: Online Shop Till You Drop With Your Friends On Plurchase

#36

Post by catch23 »

slice wrote: post #26

Isn't that borderline phishing, Seriously? I'm thinking about a naive visitor, who doesn't have an account with Plurchase. If s/he now tries to login to Zappos, Plurchase sees and handles her/his username & password. This is a simple man-in-the-middle attack.

It is somewhat man-in-the-middle, but not much different than your browser-side plugins... some browser side plugins load external javascript into your webpages so it wouldn't be significantly different. Wesabe.com (personal finance site like Mint) allows you to enter in your financial data directly into their system, and also provides a firefox plugin that will fetch that financial data should you be worried about Wesabe's security. However, few people have used their firefox plugin for doing it.

Re: Online Shop Till You Drop With Your Friends On Plurchase

#37

Post by eserorg »

immad wrote: post #33
eserorg wrote: post #32

Earlier quoted context omitted.

CGIProxy has full javascript support. It also handles cookies and can dynamically determine whether or not it should proxify images and CSS assets. CGIProxy also handles flash. In fact, CGIProxy can fully proxify an SSL-encrypted GMail session out-of-the-box. CGIProxy is _very_ mature technology. Take a look at the source code -- it handles 100's of special-cases. Some of the special-cases that CGIProxy handles are t…

It rewrites the flash and javascript so that they call your reverse proxy instead of the original domain?

Yes.

Re: Online Shop Till You Drop With Your Friends On Plurchase

#38

Post by fauigerzigerk »

I'm afraid merchants won't like this idea at all. Their domain name is hidden. Users bookmark plurchase instead of the merchant's site and there are endless opportunities to implement comparison shopping on top of it. I also wonder about SSL and certificates once it's checkout time.

This looks like an endless legal and technological battle against merchants. I'll be very intersted in the outcome because obviously wrapping sites via a proxy is a powerful technique for many things.

Re: Online Shop Till You Drop With Your Friends On Plurchase

#39

Post by hussong »

catch23 wrote: post #31
eserorg wrote: post #29

Plurchase is just a reverse HTTP proxy server looking for an application. It's not too hard to do something similar in perl. 1) start off by downloading a copy of CGIProxy: http://www.jmarshall.com/tools/cgiproxy/ 2) Setup an apache server running CGIProxy 3) Hack the interface box at the top of the screen to include: facebook integration, a meebo chat box, etc... The problem is that this sort of thing tends to _real…

Most reverse proxies do the easy stuff, but all the hard stuff is still on the client side javascript/flash code. It'll probably take you a few hours to hack up one of the existing proxies out there and get "almost" what plurchase has, but you'll soon see that menus won't render in amazon and links just won't work because the link menu system is completely dynamically constructed so most proxies won't take you this f…

I wonder how much of your resources will be bound long-term trying to keep up with ever-changing frontend code of the different shops and sites. It's something that keeps most of the teams doing javascript injecting browser plugins really busy.
Post reply on HN