Have you looked at running wireguard on your server and connecting to it from the pi. Then you have a vpn between server and the pi where you can send data in any protocol
A Raspberry Pi as a decent residential proxy
21–30 of 47 posts
Re: A Raspberry Pi as a decent residential proxy
#22Have you looked at running wireguard on your server and connecting to it from the pi. Then you have a vpn between server and the pi where you can send data in any protocol
I haven't but I'll think about it, thanks for the idea.
Mainly the custom-made proxy allows scaling the app to several pies if I ever need to.
Re: A Raspberry Pi as a decent residential proxy
#23Earlier quoted context omitted.
I once accidentally left privoxy open on one of my Linux servers and misconfigured ACL. A couple hours later it started getting hit with loads of traffic. A Google search quickly revealed that my ip:port ended up on several “free proxy” lists. They probably found it on Shodan or something. I quickly fixed acl to drop the traffic, but IIRC my server persisted on the free proxy lists for a couple days afterwards. That…
Actually, you are giving one advantage to the custom-made proxy, it's not easily discoverable by bots, nor it could be used right away.
Just using non-default settings for your proxy would be enough to thwart 99.9% of automated attacks, and you wouldn't have to rely on a hand-rolled proxy.
Re: A Raspberry Pi as a decent residential proxy
#24Earlier quoted context omitted.
Actually, you are giving one advantage to the custom-made proxy, it's not easily discoverable by bots, nor it could be used right away.
Just using non-default settings for your proxy would be enough to thwart 99.9% of automated attacks, and you wouldn't have to rely on a hand-rolled proxy.
This is the same for ssh. If you want to decrease log size, change the port used away from default.
Re: A Raspberry Pi as a decent residential proxy
#25For more reliable ssh tunnels you can use autossh.
Re: A Raspberry Pi as a decent residential proxy
#26Have you looked at running wireguard on your server and connecting to it from the pi. Then you have a vpn between server and the pi where you can send data in any protocol
I do this to expose services to the internet in a limited way. Wireguard link between a pi running, say, octoprint, and a vps. Then, the vps is running nginx or caddy as a reverse proxy over that wireguard link, giving me https access and even letting me add basic auth if I want another layer of authentication.
Re: A Raspberry Pi as a decent residential proxy
#27I run a number of news website scrapers for various projects. Originally I ran them all on AWS EC2 instances and racked up sizeable bills. Recently though, I pulled an unused Raspberry Pi from my desk drawer, installed them all onto the Pi, and now run them for near free from my desk! Granted I need to be a little bit more efficient with memory and processor cycles, but that’s a fun little challenge. I find if you re…
I did the same thing when I was running a scraper. I throttled the requests quite low to be respectful so it took a while to gather all the data, but it worked just fine.
For this article, does the OP feel they need the proxy due to scraping rapidly?
Re: A Raspberry Pi as a decent residential proxy
#28> Surprisingly, 256 MB in RAM have been enough to maintain the app running for months until now. What is total RSS of all your proxy processes?
$ free -h total used free shared buff/cache available Mem: 432Mi 194Mi 26Mi 20Mi 211Mi 164Mi Swap: 99Mi 2.0Mi 97Mi $ top Tasks: 69 total, 1 running, 68 sleeping, 0 stopped, 0 zombie %Cpu(s): 7.3 us, 4.3 sy, 0.0 ni, 88.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 432.4 total, 26.2 free, 194.7 used, 211.5 buff/cache MiB Swap: 100.0 total, 98.0 free, 2.0 used. 164.7 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TI…
Don't forget to set gpu_mem in config.txt. It looks like you are running with the default, which is 64MB for the GPU. I'm guessing you are not using the GPU in this application and should reduce this value to the bare minimum (which I don't recall, sorry)
Re: A Raspberry Pi as a decent residential proxy
#29Have you looked at running wireguard on your server and connecting to it from the pi. Then you have a vpn between server and the pi where you can send data in any protocol
I do this to expose services to the internet in a limited way. Wireguard link between a pi running, say, octoprint, and a vps. Then, the vps is running nginx or caddy as a reverse proxy over that wireguard link, giving me https access and even letting me add basic auth if I want another layer of authentication.
It's a shame that client-certificates is implemented in such a clunky way in most browsers and operating systems, because that could also be an easy way to achieve this even without installing anything.
Re: A Raspberry Pi as a decent residential proxy
#30> Don’t expose the proxy to the world or attackers will be able to interact with your home devices Or use it to get stuff on the internet that they don't want to get directly themselves. Once upon a time I accidentally enabled open proxying on the web server on my Linux box. This was before smart phones and tablets and such, when my Linux box was the only network client I had, and so I didn't have a NAT router. Just…
20 years ago, I thought it would be cool to monitor Web and other port traffic on my Linux home PC by logging it to my X root window...
That's when I realized from the scrolling log messages that even residental ADSL was already under heavy probing from around the world.
I decided I didn't have the time or the stomach to try to deal with that at home, and there wasn't much I could do, so I disabled the logging.