Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Templar: A proxy to improve HTTP API interactions

github.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Templar: A proxy to improve HTTP API interactions

#11

Post by evanphx »

simonw wrote: post #10
evanphx wrote: post #5

Earlier quoted context omitted.

Fire-and-Forget is on the todo list. Since you're asking about it I'll bump it up. The later, Templar could have an option that tell it to only connect to public ips. Would that do it for ya?

Yes, that would be excellent - do you mean just public IP ranges? One possible snag: I heard from a co-worker that if you are running on EC2 and you make an HTTP request to another site that coincidentally is also hosted on EC2 that traffic can sometimes be routed over a private IP range.

That's good point actually, I could see that happening.

What is your concern about the url used? That they point at something a user is trying to coerce you to hit? If so, that could be a public IP too..

Re: Templar: A proxy to improve HTTP API interactions

#12

Post by evanphx »

simonw wrote: post #10
evanphx wrote: post #5

Earlier quoted context omitted.

Fire-and-Forget is on the todo list. Since you're asking about it I'll bump it up. The later, Templar could have an option that tell it to only connect to public ips. Would that do it for ya?

Yes, that would be excellent - do you mean just public IP ranges? One possible snag: I heard from a co-worker that if you are running on EC2 and you make an HTTP request to another site that coincidentally is also hosted on EC2 that traffic can sometimes be routed over a private IP range.

Looking at that CPAN module, it wouldn't be hard at all to add a blacklist to Templar via a config file that is checked. The EC2 => EC2 issue is probably the biggest stumbling block to getting what you want. I'll have to investigate that, it seems like that would only happen if internal to EC2 they do ICMP redirects...

Re: Templar: A proxy to improve HTTP API interactions

#13

Post by jkarneges »

evanphx wrote: post #11
simonw wrote: post #10

Earlier quoted context omitted.

Yes, that would be excellent - do you mean just public IP ranges? One possible snag: I heard from a co-worker that if you are running on EC2 and you make an HTTP request to another site that coincidentally is also hosted on EC2 that traffic can sometimes be routed over a private IP range.

That's good point actually, I could see that happening. What is your concern about the url used? That they point at something a user is trying to coerce you to hit? If so, that could be a public IP too..

It's more about hitting internal services, where there may not be adequate protections in place. There's some earlier discussion here: https://news.ycombinator.com/item?id=7139176

Re: Templar: A proxy to improve HTTP API interactions

#14

Post by evanphx »

jkarneges wrote: post #13
evanphx wrote: post #11

Earlier quoted context omitted.

That's good point actually, I could see that happening. What is your concern about the url used? That they point at something a user is trying to coerce you to hit? If so, that could be a public IP too..

It's more about hitting internal services, where there may not be adequate protections in place. There's some earlier discussion here: https://news.ycombinator.com/item?id=7139176

Ah! Ok, I got it. Feels like the right way to handle this is the allow a blacklist to be defined via config file, then applied as request, something like "X-Templar-Blacklist: internal". The list would be a set of ip ranges and thus you'd have to construct the list so that the EC2 => EC2 problem doesn't crop up, but it's doable!

Re: Templar: A proxy to improve HTTP API interactions

#16

Post by jfindley »

The HTTPS implementation looks dangerous at first sight - it looks like an attacker could very easily just strip the X-Templar-Upgrade out from the request, and then end up with everything being un-encrypted - without the client throwing any warnings.

Could you possibly provide more details of how you avoid this?

Re: Templar: A proxy to improve HTTP API interactions

#19

Post by aaronem »

evanphx wrote: post #9
aaronem wrote: post #7

Maybe I'm missing the point here, but I'm not entirely sure I see the point here. What can Templar do for me that, say, nginx can't? All else equal, if nginx is already in my stack and Templar is not, why do I want to adopt Templar, or indeed even look twice at it?

A great question. Nginx is not typically configured as a normal proxy but it can certainly do it. A difference is that Templar gives you control of options applied to each request differently rather than the same the same to all. From timeouts to caching, one Templar can be used for all different kinds of upstream APIs.

Well, if it's worth adding a whole new layer to my stack to deal with this kind of stuff, surely it's even more worth extending an existing layer to deal with it; as a corollary, if I know enough about the problems I'm having to know what I need Templar to do for me, then I know enough about them to know what I need nginx to do for me, too.

Again, though, maybe I'm just missing the point here. Perhaps it would help to know what use case prompted Templar's development.

Re: Templar: A proxy to improve HTTP API interactions

#20

Post by falcolas »

aaronem wrote: post #19
evanphx wrote: post #9

Earlier quoted context omitted.

A great question. Nginx is not typically configured as a normal proxy but it can certainly do it. A difference is that Templar gives you control of options applied to each request differently rather than the same the same to all. From timeouts to caching, one Templar can be used for all different kinds of upstream APIs.

Well, if it's worth adding a whole new layer to my stack to deal with this kind of stuff, surely it's even more worth extending an existing layer to deal with it; as a corollary, if I know enough about the problems I'm having to know what I need Templar to do for me, then I know enough about them to know what I need nginx to do for me, too. Again, though, maybe I'm just missing the point here. Perhaps it would help t…

In my opinion, the draw is the ability to use it when connecting to third party endpoints (so you can't control how they cache or how they respond) with third party libraries (which means you lack fine grained control over timeouts and caching on your end).

Personally, I'd rather gain the necessary control through the request mechanisms, but for a quick-and-dirty solution, this would probably work pretty well.

Post reply on HN