Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Bt – BitTorrent library in Java 8

github.com

Post reply on HN
Print view View original

51–60 of 89 posts

Re: Show HN: Bt – BitTorrent library in Java 8

#51

Post by pjmlp »

weberc2 wrote: post #14
atomashpolskiy wrote: post #13

Earlier quoted context omitted.

I'm using Lanterna, a Java library for creating text-based terminal GUIs ( https://github.com/mabe02/lanterna ), and the startup is 1-2 seconds. Can't recommend it enough

Good to know, but 1-2 seconds is still very slow compared to a native app. I have an 8MB Go CLI application that can startup and print its help in .05 seconds, and even that is probably slow compared to a C or Rust equivalent.

Any savy Java developer knows how to AOT compile to native code, if that really matters.

There are plenty of JDKs that support it, and even Oracle is finally adding support for it with Java 9, initially only for Linux x64.

By Java 10 timeframe no one that only knows about OpenJDK and not the several other JDKs can state that Java starts slow vs C, because both will be AOT compiled to native code, loading native .so files.

Re: Show HN: Bt – BitTorrent library in Java 8

#52

Post by pjmlp »

pebers wrote: post #34
deepsun wrote: post #17

Earlier quoted context omitted.

Well, printing stuff is easy. Just tried: $ time java HelloWorld Hello, World real 0m0.088s user 0m0.088s sys 0m0.016s

That's presumably not loading any libraries at all though - the time to being interactive (or just doing something useful) will get worse as more bytecode has to be decompressed and loaded.

> That's presumably not loading any libraries at all though

Sure it is loading libraries, everything that System.out.println() depends on.

Re: Show HN: Bt – BitTorrent library in Java 8

#53

Post by pjmlp »

weberc2 wrote: post #32
deepsun wrote: post #15

Earlier quoted context omitted.

Some apps just include JRE into the app package (+100MB), and just use it if it's not provided by OS. But then you need to provide different app packages for different OSes.

C, Go, and Rust would require different app packages for different OSes--that's not a problem. The problem is bundling a 100MB runtime for a CLI.

Java 9 has a linker, you have remove everything that is not needed and have a application specific runtime.

On top of that, if lucky enough to be on an Linux x64 system, you can AOT compile to native code.

Re: Show HN: Bt – BitTorrent library in Java 8

#54

Post by atomashpolskiy »

olalonde wrote: post #49
atomashpolskiy wrote: post #45

Earlier quoted context omitted.

I'd say something in the range of 500-1000 hours.

That's less than I was expecting. Did you study any existing library to get up to speed?

No, after I had studied BEP-3, wiki and and a few papers (most importantly "BitTorrent economics"), I had a very clear understanding of what needs to be done.

Re: Show HN: Bt – BitTorrent library in Java 8

#55

Post by atomashpolskiy »

silotis wrote: post #46
atomashpolskiy wrote: post #44

Earlier quoted context omitted.

I have several hundred unit tests and also a bunch of integration tests. While UTs usually test API of individual classes, each of the ITs creates a swarm of local peers and launches a torrenting session with certain conditions: seeders to leechers ratio, downloading from .torrent file vs using a magnet link, PEX enabled/disabled, encrypted vs raw message streams, etc. Benchmarks (like in libtorrent) is something I'm…

I would advise against implementing a custom cache for piece data. Caching file data in-app is counterproductive because it wastes memory duplicating data which the OS already holds in its page cache. Instead memory mapped I/O should be preferred.

It makes sense, what would you advise in case of big files that don't fit into memory (or when it's undesirable to use big amounts of RAM, and speed can be sacrificed)? Maybe caching should be disabled by default but with the option to turn it on if the user wants it?

Re: Show HN: Bt – BitTorrent library in Java 8

#56

Post by atomashpolskiy »

segmondy wrote: post #50

Amazing work, how long did it take to build? Where you working on it full-time or part-time?

Thank you! Calendar year, in the evenings and on weekends, with sprints of several weeks and a month or two of rest/AFK after each sprint to keep sanity (I have a full-time job, a wife and a 3 year old kid). Happy that I had the persistence to keep working on the project

Re: Show HN: Bt – BitTorrent library in Java 8

#57

Post by atomashpolskiy »

pjmlp wrote: post #51
weberc2 wrote: post #14

Earlier quoted context omitted.

Good to know, but 1-2 seconds is still very slow compared to a native app. I have an 8MB Go CLI application that can startup and print its help in .05 seconds, and even that is probably slow compared to a C or Rust equivalent.

Any savy Java developer knows how to AOT compile to native code, if that really matters. There are plenty of JDKs that support it, and even Oracle is finally adding support for it with Java 9, initially only for Linux x64. By Java 10 timeframe no one that only knows about OpenJDK and not the several other JDKs can state that Java starts slow vs C, because both will be AOT compiled to native code, loading native .so f…

Good point, but I wonder what size will the final binaries have, given that bytecode inflates significantly when translated to the machine code (there was a good discussion on HN recently about app sizes)

Re: Show HN: Bt – BitTorrent library in Java 8

#58

Post by pjmlp »

atomashpolskiy wrote: post #57
pjmlp wrote: post #51

Earlier quoted context omitted.

Any savy Java developer knows how to AOT compile to native code, if that really matters. There are plenty of JDKs that support it, and even Oracle is finally adding support for it with Java 9, initially only for Linux x64. By Java 10 timeframe no one that only knows about OpenJDK and not the several other JDKs can state that Java starts slow vs C, because both will be AOT compiled to native code, loading native .so f…

Good point, but I wonder what size will the final binaries have, given that bytecode inflates significantly when translated to the machine code (there was a good discussion on HN recently about app sizes)

In any case smaller than Electron apps.

Re: Show HN: Bt – BitTorrent library in Java 8

#59

Post by the8472 »

atomashpolskiy wrote: post #55
silotis wrote: post #46

Earlier quoted context omitted.

I would advise against implementing a custom cache for piece data. Caching file data in-app is counterproductive because it wastes memory duplicating data which the OS already holds in its page cache. Instead memory mapped I/O should be preferred.

It makes sense, what would you advise in case of big files that don't fit into memory (or when it's undesirable to use big amounts of RAM, and speed can be sacrificed)? Maybe caching should be disabled by default but with the option to turn it on if the user wants it?

you can memory-map files in chunks and keep references to the mapped buffers or evict them once a certain limit is reached.

note that mapped buffers are not necessarily backed by RAM, they act more like swap space, except it goes straight to the original file instead of a swap partition. the downside is that anything reading from the buffer can unpredictably incur IO overhead if there's memory pressure and the OS decides to not back them with memory. java doesn't have an api to check residency of buffers. a possible workaround is to first queue them up on a separate thread and forcing them in before using. if they're already there it should have low latency. if they aren't then only the victim thread eats the IO penalty.

Post reply on HN