51–60 of 115 posts
It is a new project, it enables your server to use all other front end and back end technologies at the same time! That is what makes it so fantastic!
You get all front end JavaScript frameworks, and all back end server technologies, all for one low price. Just install this new component into your project. It will download the other few gigabytes and hook it into your project.
I have a personal project for 5 years in this area https://goreplay.org, and investigating ways to automatically find issues in web applications. Project get traction, but I do not see any competition so far. And there is a lot of reasons for this because in such under-researched areas being a pioneer for both project owner and end user require them a different set of mind. And this is really hard to market.
A workload-aware data distribution proxy. Let me explain. A lot of people talk about multi-cloud these days. Either AWS and Azure, or AWS and their own datacenter, or whatever. While it's really easy to send compute jobs to one cloud or the other based on which one is best suited for the job, the big issue is that your compute job will need data. Right now your choices are to have a copy of all your data in all the c…
A workload-aware data distribution proxy. Let me explain. A lot of people talk about multi-cloud these days. Either AWS and Azure, or AWS and their own datacenter, or whatever. While it's really easy to send compute jobs to one cloud or the other based on which one is best suited for the job, the big issue is that your compute job will need data. Right now your choices are to have a copy of all your data in all the c…
One project that is aiming in this direction: http://seaclouds-project.eu/
The tool inspects the source code and wraps calls across processes (no IDL).
I wrote a (beginning of) a description here: http://dpt.slasheva.com/project-ideas.html#middleware
I want a distributed (not cloud) file system that handles many files and WAN latency (i.e. not HDFS). It might be a cross between Git and BitTorrent. It feels like deb repos, PyPI, NPM, CPAN, CRAN, etc. should be put in there, with the addition of binaries for popular architectures. And probably Docker-like images, although I think if they are not opaque blobs, it would be better for rsync-like differential compressi…
Basically you want S3, which has BitTorrent BTW. S3 has multiple widely used command line tools that support syncing (cloning, pushing). There are multiple heavily used S3-like systems, open and closed source, some are S3 API compatible. Examples include Ceph and OpenStack Swift. The reason people use cloud services instead of running these themselves is because the cloud storage fee is usually less than paying someo…
Plenty of people run their own BitTorrent trackers too.
In my mind, "cloud" open source means: you need a team of experts to run it. git and BitTorrent are different -- they are designed for you to run it yourself.
Also, the model is to "sync" and then "read/write", as with git (and BitTorrent). Not just read/write remotely. So maybe I should call it a "replicated file system" rather than a distributed one.
I guess the main difference with git is that it should handle large binaries / many files, you shouldn't have to clone the entire repo, and maybe users/permissions.
I want a distributed (not cloud) file system that handles many files and WAN latency (i.e. not HDFS). It might be a cross between Git and BitTorrent. It feels like deb repos, PyPI, NPM, CPAN, CRAN, etc. should be put in there, with the addition of binaries for popular architectures. And probably Docker-like images, although I think if they are not opaque blobs, it would be better for rsync-like differential compressi…
Upspin? https://upspin.io/
Still, I'm definitely interested in whether I could use it to upload say 1 TB of source code, and maybe 5 TB of binaries, and have people sync efficiently (and partially).
I want a distributed (not cloud) file system that handles many files and WAN latency (i.e. not HDFS). It might be a cross between Git and BitTorrent. It feels like deb repos, PyPI, NPM, CPAN, CRAN, etc. should be put in there, with the addition of binaries for popular architectures. And probably Docker-like images, although I think if they are not opaque blobs, it would be better for rsync-like differential compressi…
A few random thoughts: IPFS sounds a lot like what you want. Tahoe LAFS plays in this space a bit The newer distributed file systems I've seen don't like permissions. They like cryptographically-backed capabilities. You have the permission to read the file because you have the ability to decrypt, through the key. (Of course, key management is easy ). Some user-facing distributed filesystem talks drift into FUSE (or s…
I don't think anyone has solved the issues raised in Out of the Tar Pit in the data storage space. We still spend a ton of effort, perhaps more than ever before, wrangling incidental complexity. In 2018, I should be able to define a simple relational schema in 30 seconds and start using it with effectively zero constraints around data access patterns, transaction volume, and data scale, and with no design decisions n…
I would use it for replicated / distributed storage for data frames (e.g. R or Pandas), which is somewhat related to my other request [1]
The difference here is that there is structure to the files.
An open-source platform for self-hosting function-as-a-service - something that provides the tooling for easily saying "deploy this function, auto scale it, route HTTP traffic to it, now atomically replace it with this new version" without having to lock yourself in to Google/AWS/Azure.
I'm sure I've seen announcements for others pop up on HN too.