Postgres
Redis
Express + knexjs (TS on backend)
Heroku
Create React App front end.
Simple, boring technologies.
171–180 of 327 posts
Postgres
Redis
Express + knexjs (TS on backend)
Heroku
Create React App front end.
Simple, boring technologies.
I run a file sharing / content delivery platform called pixeldrain: https://pixeldrain.com The system serves 4 PB of data to 60 million visitors per month. I have served 30 PB and 700 million file views since I started tracking usage somewhere in 2018. I'll go from front to back: - Most of the frontend is plain HTML, CSS and JS. I have started transitioning some pages to Svelte. I like this framework for its speed an…
I run a file sharing / content delivery platform called pixeldrain: https://pixeldrain.com The system serves 4 PB of data to 60 million visitors per month. I have served 30 PB and 700 million file views since I started tracking usage somewhere in 2018. I'll go from front to back: - Most of the frontend is plain HTML, CSS and JS. I have started transitioning some pages to Svelte. I like this framework for its speed an…
I'd expect that the file access patterns are power-law distributions, i.e. recently uploaded files are requested more often than older files. If that's the case, can you use this property for sharding by having hot and cold storage servers?
How do you handle users uploading forbidden content? I see from another comment that you ban the usual types of illegal content. But in practice, do you manually review every mail you get on your abuse contact and take appropriate action? What's the most common type of complaint?
From a business perspective: How did you grow your site? I imagine the competition must be rough since file hosting is such a "simple" service.
The best bang for the time + effort that I’ve seen so far.
I run a file sharing / content delivery platform called pixeldrain: https://pixeldrain.com The system serves 4 PB of data to 60 million visitors per month. I have served 30 PB and 700 million file views since I started tracking usage somewhere in 2018. I'll go from front to back: - Most of the frontend is plain HTML, CSS and JS. I have started transitioning some pages to Svelte. I like this framework for its speed an…
Hi! I had never heard of your service, so I googled it and accidentally misspelled "Pixel" as I often do as "Pixle". The first result was a reddit thread and the second seemed to be a full-length rip of Becoming Royal that had some really malicious ads which were clickjacking and covering the content. Just thought you might like to know. :)
Could you maybe send me a screenshot / screen capture of the ads and send them to support@pixeldrain.com? I'd really appreciate that.
I taught myself to program to build [redacted], a comparison website for phone contracts in the UK. Front end: plain JavaScript Back end: Node.js Host: $10 DigitalOcean droplet Web server: Nginx SSL: Let's Encrypt with certbot Proxy: Cloudflare Database: Postgres Caching: Redis Indexing: MeiliSearch I wanted to learn JavaScript and understand the fundamentals, so I used vanilla JavaScript for the front end without an…
https://flat.social - a webapp for building playful virtual spaces for meetings and socials. (demo here: https://www.youtube.com/watch?v=JO9o1DobMzs ) Front-end: Typescript, Next.js, Styled Components Back-end: Typescript, Node.js, Express, Socket.io, Express Audio/video: Mediasoup Deployment: Github workflows, Docker Analytics: GA, Mixpanel, Sentry Hosting: A couple CPU-optimised droplets from Digital Ocean, Firebas…
Earlier quoted context omitted.
Hi! I had never heard of your service, so I googled it and accidentally misspelled "Pixel" as I often do as "Pixle". The first result was a reddit thread and the second seemed to be a full-length rip of Becoming Royal that had some really malicious ads which were clickjacking and covering the content. Just thought you might like to know. :)
Thanks for letting me know. I suspect my advertising partner is doing some shady business on my site. I have been trying to catch them but I never seem to get the malicious ads myself, even when using a VPN from more eastern or southern nations. Could you maybe send me a screenshot / screen capture of the ads and send them to support@pixeldrain.com? I'd really appreciate that.
I run a file sharing / content delivery platform called pixeldrain: https://pixeldrain.com The system serves 4 PB of data to 60 million visitors per month. I have served 30 PB and 700 million file views since I started tracking usage somewhere in 2018. I'll go from front to back: - Most of the frontend is plain HTML, CSS and JS. I have started transitioning some pages to Svelte. I like this framework for its speed an…
You mention that you have user-facing servers as well as storage servers. So do the user-facing servers act as reverse proxies for the storage servers, or do you simply serve a redirect? I'd expect that the file access patterns are power-law distributions, i.e. recently uploaded files are requested more often than older files. If that's the case, can you use this property for sharding by having hot and cold storage s…
Yes, my user-facing servers are proxying the files to the users.
> I'd expect that the file access patterns are power-law distributions
Exactly. On each server I have a sorted slice in-memory which keeps track of how often files are being requested. The most popular files are cached and files which drop out of the cache are moved to HDD storage servers in Helsinki. This way I can serve 10 Gbps of data to the users while only putting a 1 Gbps load on the storage nodes.
> How do you handle users uploading forbidden content?
I get a lot of abuse mails, mainly copyright violations. I have a mailserver which is hooked into pixeldrain which scans mails from common copyright offices and automatically blocks the files. For other types of abuse I have a report button on the download page. Users report content which breaks the rules and once the number of reports hits a certain threshold the file is blocked.
> From a business perspective: How did you grow your site?
Most file sharing sites are really terrible. Like almost unusable. It's pretty easy to beat the competition in UX. I just started using pixeldrain myself on reddit and other forums. In the beginning people complained in the comments about the ergonomics of the site and I listened carefully. Eventually when the kinks were worked out other people started using it too.
* We use cloudflare workers for our API endpoints which have given us amazing reliability and scalability (which is of course very important for our service)
* We use cloudflare KV and cloudflare cache for some caching and logged in user sessions.
* Mailgun, FaunaDB, Sentry, LogDNA, Stripe, BigQuery.
* The web app is good old server side rendered html and css with a tiny bit of JS here and there.
* The widget is open source, written in Typescript (and as it has to run in really old browsers there's Babel to transpile. The solver inside of it and the proof of work library is AssemblyScript (i.e. WASM), with a JS fallback.
* PHP for our wordpress plugin.
The way we operate is that we provide free (or very low cost) plans for hobby users and stuff like small (wordpress) blogs to hopefully make a dent into recaptcha's market share, it's a bit of a social mission. Larger companies pay for more advanced protection features, EU-only endpoints, (as well as custom agreements and other paperwork). That balance has worked well for us, and even the small customers that use the free service contribute to our protection.
The past half year or so we learned that our customers that bring in the lion share of the revenue really prefer it if we keep their processing and data in Europe (our privacy friendliness and our EU-basedbess are big selling points, even more than improved ux+accessibility). So much so that we are heavily investing into that, and we are slowly moving in favor of our own infra in Hetzner (Germany).
There our tech stack is fully Golang (Fiber framework), with Redis, Postgres and Clickhouse as data stores. The way that our system works is that we look at patterns of access, and we can tweak the difficulty of the proof of work challenge on a request by request basis. One nice property is that it's not all or nothing: if we suspect a puzzle request is from a spammer they will get a rather difficult puzzle which will take a while to solve, but at least it won't lock out any false positives. Clickhouse is fantastic for this purpose (putting in millions of events is not even close to its capacity, it's lightning fast). Of course the widget itself also has the most basic of anti headless browser checks, but that will only deter the most naive spammers.
Of course no captcha system is perfect and will protect against a spammer who is willing to spend real resources (e.g. pay compute, or human labelers) to spam, but so far we're happy with its effectiveness, and it warms our hearts when we receive messages from blind or even deaf-blind users that encountered our captcha and web out of their way to say thank you :). I hope that at some point captcha labeling tasks can be a thing of the past.