Live data from Hacker News

IPFS 0.5

blog.ipfs.io

31–40 of 145 posts

Re: IPFS 0.5

#31

I have a Go project that imports IPFS and libp2p libraries. What's the recommended way of upgrading those dependencies?

The release notes contain a pretty extensive list of what APIs have changed. The path to upgrade the dependencies is probably to run `go get -u ` for the direct dependencies you're including, and then fixing errors that pop up from doing so.

Depends what exactly you are importing and for what. I suggest you ask in the forums and post a link to your project. go get -u is dangerous as it updates all subdependencies used to latest.

Usually take go-ipfs's go.mod as a guide on what versions to use.

Re: IPFS 0.5

#32
post #10

Earlier quoted context omitted.

We do use Kademlia - but note this release actually runs 2 DHTs - one for LAN connections and one for WAN connections. You could easily imagine a small Mars outpost as the LAN DHT where you can do fast retrieval for all content already available locally without hitting interplanetary lookup times. https://docs-beta.ipfs.io/recent-releases/go-ipfs-0-5/featur...

What are your plans WRT Coral? I noticed there's a work-in-progress Go implementation. Is it going to supercede Kademlia eventually?

Yeah, thats roughly the idea. We can evolve the current DHT into something coral-like, or if something better comes along, we will use that.

Re: IPFS 0.5

#33

I wish they hadn't picked go. Go isn't a bad language really, but it's a huge pain to integrate a go library into an app written in another language.

Go can handle 100k goroutines (green threads) without a sweat. This is useful for p2p systems in which you are connected to thousands of peers multiplexing several p2p protocols with each concurrently. You want to be able to switch threads as fast as possible. The concurrency model is a bit easier to reason about too. It would be good to see how Rust and others do. Other languages excel at other things (i.e. embedded…

Rust can handle the same thing though the typical model now is using async-await and scheduling tasks on a threadpool. While I won't say its conceptually as simple as go can be, Rust also has channels, mutexes, and other concurrency primitives that allow you to create a similar story. It will also end up being more performant and less work to make correct (typically) due to its small runtime and extensive compile-time checking.

Re: IPFS 0.5

#34
post #26

This uses DHTs. How resilient is it against Sybil attacks? Also how does it work under global netsplit conditions, like if someone borks or attacks BGP in such a way that 1/3 of the world is not reachable? My impression is that DHTs fall down pretty hard under the latter scenario and are also pretty vulnerable to the Sybil scenario if the attacker has enough resources to mount a really serious attack. They're okay fo…

Totally agree with you. This is an issue for most open membership p2p networks. Effectively, you need to create some type of reputation system or barrier to accruing power to make it harder for attackers to DOS/manipulate the network than for well-behaving nodes to operate it. IPFS definitely thinks about resistance to a variety of attacks like the ones you describe - but we still have a ways to go. Next steps might look like improvements to the DHT, or other more selective content routing solutions (ex, expanding on our dht-client vs dht-server designation to make content routers more selective/reputation-based).

Check out our most recent research discussion for some thoughts about how we might scale up Sybil resistance in p2p networks: https://www.youtube.com/watch?v=L4SJzoKHKPk

Re: IPFS 0.5

#35

I wish they hadn't picked go. Go isn't a bad language really, but it's a huge pain to integrate a go library into an app written in another language.

Same thing that turned me off. There’d have been lots of cool and useful integrations popping up “for free” all over the place if they’d written it in a good library/integration language. Good language for a daemon, bad language for the reference (so, for anything halfway complicated, usually only) implementation of something intended to become a foundational protocol.

Re: IPFS 0.5

#36
post #33

Earlier quoted context omitted.

Go can handle 100k goroutines (green threads) without a sweat. This is useful for p2p systems in which you are connected to thousands of peers multiplexing several p2p protocols with each concurrently. You want to be able to switch threads as fast as possible. The concurrency model is a bit easier to reason about too. It would be good to see how Rust and others do. Other languages excel at other things (i.e. embedded…

Rust can handle the same thing though the typical model now is using async-await and scheduling tasks on a threadpool. While I won't say its conceptually as simple as go can be, Rust also has channels, mutexes, and other concurrency primitives that allow you to create a similar story. It will also end up being more performant and less work to make correct (typically) due to its small runtime and extensive compile-tim…

Maybe if IPFS had started today they would have picked Rust.

Honestly I'm not sure why Go was picked specificall, but if you put things in the context of 5 years ago it doesn't seem a crazy choice.

Re: IPFS 0.5

#37

I wish they hadn't picked go. Go isn't a bad language really, but it's a huge pain to integrate a go library into an app written in another language.

They also picked javascript ¯\_(ツ)_/¯

I used to hate JS, but honestly, it made a lot of really smart decisions for dynamic documents. Some decisions weren’t great, but somehow they didn’t go as far wrong as C++ managed to in the same era…

Re: IPFS 0.5

#38

I wish they hadn't picked go. Go isn't a bad language really, but it's a huge pain to integrate a go library into an app written in another language.

They also picked javascript ¯\_(ツ)_/¯

JS has a better library-integration story than Go does, since there are so many JS implementations in other languages. Go drags all its runtime threads along with it.

But fair point :)

Re: IPFS 0.5

#39
post #26

This uses DHTs. How resilient is it against Sybil attacks? Also how does it work under global netsplit conditions, like if someone borks or attacks BGP in such a way that 1/3 of the world is not reachable? My impression is that DHTs fall down pretty hard under the latter scenario and are also pretty vulnerable to the Sybil scenario if the attacker has enough resources to mount a really serious attack. They're okay fo…

For BGP partitions / Significant network outage conditions: You may not find content partitioned on the other side of a netsplit from you, but the degraded condition is that you can still query and find content present in the same part of the network as you. This is better than centralized solutions face at present - e.g. google not being available in China, and isn't too far off from what you might hope. For Sybils:…

> You may not find content partitioned on the other side of a netsplit from you, but the degraded condition is that you can still query and find content present in the same part of the network as you.

This becomes hugely problematic the minute you start using IPNS. On one side of the split, the name `foo` can resolve to `bar`, but on the other side, it resolves to `baz`. If you're trying to impersonate someone, then a netsplit would make it easy for you to do so (barring an out-of-band content authentication protocol, of course).

> You've left the attack you're worried about pretty vague. IPFS itself doesn't need to tackle many of the sybil-related issues by being content addressable (so only worrying about availability - not integrity) and not being a discovery platform - so not worrying about spam / influence.

On the contrary, a Sybil node operator can censor arbitrary content in a DHT by inserting their own nodes into the network that are all "closer" to the targeted key range in the key space than the honest nodes. This can be done by crawling the DHT, identifying the honest nodes that route to the targeted key range, and generating node IDs that correspond to key ranges closer than them. Honest nodes will (correctly) proceed to direct lookup requests to the attacker nodes, thereby leading to content censorship.

Honest nodes can employ countermeasures to probe the network in order to try and see if/when this is happening, but an attacker node can be adapted to behave like an honest node when another honest node is talking to it.

> For the remaining degradation attacks - someone overwhelming the DHT with misbehaving nodes - there's been a bunch of work in this release looking at how to score peers and figure out which ones aren't worth keeping in the DHT.

Sure, and while this is a good thing, it's ultimately an arms race between the IPFS developers and network attackers who can fool the automated countermeasures. I'm not confident in its long-term ability to fend off attacks on the routing system.

Re: IPFS 0.5

#40
post #18

I wish they hadn't picked go. Go isn't a bad language really, but it's a huge pain to integrate a go library into an app written in another language.

C or Rust would be perfect. No GC runtime, no name mangling, fast, great ffi story. Rust needs to start edging out Go for new tools. Kubernetes, Envoy, IPFS, etc. would have benefitted from it. It might not have been time four years ago, but it's time now.

Keybase and KBFS as well.
Post reply on HN