Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IPFS Project Roadmap

github.com

Post reply on HN
Print view View original

111–120 of 154 posts

Re: IPFS Project Roadmap

#111

Post by justincormack »

anandology wrote: post #57

In addition to apt and npm, I would like to see docker image distribution powered by IPFS. It really feels stupid to pull images from a central registry sitting on the other side of the globe when the image is already present in the next node in your kubernetes cluster.

There is a big issue, that ipfs uses its own hash mechanism (hash of protobuf of dag), while registries (and most other existing content based distribution mechanisms) use sha256 hashes of the whole content.

eg see https://github.com/ipfs/notes/issues/269

So you can't simply interop between the two without some sort of lookup to convert hash functions. It is hugely frustrating, as basically different content based distribution mechanisms cant work together.

In theory docker image registries support pluggable hash functions, although it is not clear to me that the ipfs function is even very well defined outside its own code. We could start to add a second hash calculation to every registry operation, but it would be a performance hit which some users would not like.

(tree based content hashes that allow parallelisation are nice, but the ipfs one is very ipfs specific and more complex than it needs to be I think).

Re: IPFS Project Roadmap

#112

Post by arianvanp »

sideeffffect wrote: post #103
fwip wrote: post #18

"2019 Goal: The most used code and binary Package Managers are powered by IPFS." That's kind of stupid-ambitious for 2019 when another 2019 goal is "a production-ready implementation" and IPFS has been around for 3 years already. This isn't a roadmap, it's a wishlist. And I'm someone who wants to see IPFS succeed.

It's a bit sad, that Dhall (programmable configuration language for YAML & co.) used to use IPFS for its source/packages, but stopped, because of reliability :( (I'm wondering if there are/were others?) > Early on in the language history we used IPFS to distribute the Dhall Prelude, but due to reliability issues we’ve switched to using GitHub for hosting Dhall code. http://www.haskellforall.com/2019/01/dhall-year-in-…

We conducted initial experiments with moving nix package manager to ipfs, but also ran into too many issues.

Re: IPFS Project Roadmap

#113

Post by xwvvvvwx »

arianvanp wrote: post #112
sideeffffect wrote: post #103

Earlier quoted context omitted.

It's a bit sad, that Dhall (programmable configuration language for YAML & co.) used to use IPFS for its source/packages, but stopped, because of reliability :( (I'm wondering if there are/were others?) > Early on in the language history we used IPFS to distribute the Dhall Prelude, but due to reliability issues we’ve switched to using GitHub for hosting Dhall code. http://www.haskellforall.com/2019/01/dhall-year-in-…

We conducted initial experiments with moving nix package manager to ipfs, but also ran into too many issues.

Are these issues documented somewhere, would love to learn more. At first glance nix and ipfs seem like a perfect match

Re: IPFS Project Roadmap

#114

Post by Grustaf »

miguelrochefort wrote: post #56
viksit wrote: post #26

One of the biggest challenges with IPFS in my mind is the lack of a story around how to delete content. There may be a variety of reasons to delete things, - Old packages that you simply don't want to version (think npm or pip) - Content that is pirated or proprietary or offensive that needs to be removed from the system But in its current avatar, there isn't an easy way for you to delete data from other people's IPF…

Data should never be deleted. Ever. Start by accepting that, and everything starts to make sense.

That’s just a dogmatic statement, you need to motivate it.

Re: IPFS Project Roadmap

#115

Post by nonsens3 »

caprese wrote: post #107
nonsens3 wrote: post #21

Earlier quoted context omitted.

Swarm and IPFS together with Filecoin try to address the same problem - persistent data storage in a decentralised network. Swarm is not at all "working already" - the incentivisation layer for nodes to store data for other users is not implemented and currently mostly theoretical and work-in-progress. IPFS is more mature in comparison to Swarm, but the underlying architecture is rather different.

What is Swarm's intended incentivisation layer and where can I read about their plans? It seems like all documentation including plans are outdated, and I was ignored in their gitter chatroom where devs wanted to talk about dev things and outreach people seemed nonexistent. I see things being stored on Swarm without incentives, like plain text

Swarm documentation might not be perfect, but it is not outdated - https://swarm-guide.readthedocs.io

I believe the chapters about PSS, Swarm Feeds, ENS, Architecture, among others, are mostly up-to-date.

You can read about the incentivisation layer at https://swarm-gateways.net/bzz:/theswarm.eth/ethersphere/ora...

Currently incentivisation is not integrated or implemented in Swarm, so a user has no guarantees about what happens with their uploaded content. If the node hosting it disconnects from the network, it will be gone. The plans to address this are through the sw^3 protocols suite and/or erasure coding.

Regarding plain text - it doesn't really matter what bytes you store in Swarm - encryption is implemented and you can store non-encrypted or encrypted bytes, this has nothing to do with incentives for persistent storage.

We try to do outreach and answer community questions when possible, but the team is not big and this is currently done on a best-effort base, we could definitely improve on that front, I agree.

Re: IPFS Project Roadmap

#116

Post by sideeffffect »

fwip wrote: post #78
genpfault wrote: post #73

Earlier quoted context omitted.

> Julie's Webcast - Episode 3, with Russian subtitles > peers and seeders from both distributions can share data for the shared content So does IPFS have "plugins" for different archive/container formats so it can "see" that the underlying video/audio streams are identical between "Julie's Webcast - Episode 3.mp4" and "Julie's Webcast - Episode 3, with Russian subtitles.mkv"? Otherwise container stream interleaving w…

Last I checked it was dumb. Possibly breaking block boundaries based on a rolling hash.

https://github.com/ipfs/go-ipfs-chunker

> go-ipfs-chunker provides the Splitter interface. IPFS splitters read data from a reader an create "chunks". These chunks are used to build the ipfs DAGs (Merkle Tree) and are the base unit to obtain the sums that ipfs uses to address content.

> The package provides a SizeSplitter which creates chunks of equal size and it is used by default in most cases, and a rabin fingerprint chunker. This chunker will attempt to split data in a way that the resulting blocks are the same when the data has repetitive patterns, thus optimizing the resulting DAGs.

I think they should use the rolling hash based chunking by default

https://github.com/ipfs/go-ipfs-chunker/issues/13

Re: IPFS Project Roadmap

#117

Post by xur17 »

rnhmjoj wrote: post #104
fwip wrote: post #18

"2019 Goal: The most used code and binary Package Managers are powered by IPFS." That's kind of stupid-ambitious for 2019 when another 2019 goal is "a production-ready implementation" and IPFS has been around for 3 years already. This isn't a roadmap, it's a wishlist. And I'm someone who wants to see IPFS succeed.

I would love to see IPFS used in projects like Nix, but in the current state it's downright impossibile. I have seen IPNS taking between 5-10 minutes to solve a single address and go-ipfs with a few pinned files taking more that 5GB of memory after running for a day.

Experiencing this performance issue with ipns was a major turn off.

Re: IPFS Project Roadmap

#118

Post by nmca »

pdxww wrote: post #110

This is not a roadmap, but rather a wishlist. There is a fundamental problem that IPFS needs to solve first. This problem is called an efficient WebRTC-based DHT. In order to change the web, IPFS needs to become usable in the browsers. Since the backbone of IPFS is DHT, there need to be an efficient UDP-based solution for "DHT in the web". Right now this isn't possible and reasons are not just technical, but politica…

I actually wrote a DHT that operated over WebRTC itself with in-band signalling for my undergrad thesis, in the application/js layer. Total PITA, but a ... "good?" learning experience.

Re: IPFS Project Roadmap

#119

Post by ahnick »

pertsix wrote: post #49
viksit wrote: post #26

One of the biggest challenges with IPFS in my mind is the lack of a story around how to delete content. There may be a variety of reasons to delete things, - Old packages that you simply don't want to version (think npm or pip) - Content that is pirated or proprietary or offensive that needs to be removed from the system But in its current avatar, there isn't an easy way for you to delete data from other people's IPF…

why not encrypt it? that way it's junk data to anyone that can't decrypt it

What happens when that encryption method becomes obsolete and broken? Now this private encrypted data is accessible all over the world.

Re: IPFS Project Roadmap

#120

Post by theamk »

caprese wrote: post #108
theamk wrote: post #77

Earlier quoted context omitted.

Does Swarm’s closeness with the Ethereum means it will be unsuitable for any tasks which do not rely on financial incentives? For example, if in the future debian is moved to IPFS, then many organizations are likely run local IPFS servers with Debian repos pinned. But if debian is moved to Swarm, I do not think that many organizations will be incentivized - the money are insignificant in the total spending, while eng…

Why do you think that it would be different? "If Debian is moved to Swarm, then many organizations are likely to run local Swarm servers with Debian repos pinned"

Because from what I understand, Swarm requires Ethereum and a constant money flow. And the monthly amount of money is hard to predict. In many places I worked at, anything money related has huge overhead from finance department, and something more complex than “$X/mo” will require an immense amount of coordination and permissions.
Post reply on HN