Earlier quoted context omitted.
> Some customers (e.g. DOD) sent in clear, unsigned. They trusted their networks. Wow. I had no idea. This may be one of the dumbest things I've ever read. Encryption is free.
Didn't use to be. VIA/centaur was the first to add AES instructions, later picked up by Intel and AMD. Its not free, but not expensive. A brilliant woman from, where, Slovenia? figured a way to use SSE instructions to encrypt just as fast as the built-in instructions, I think 2 cycles per byte. I guess DJB's ciphers run much the same way, nowadays.
A Desperate Plea for a Free Software Alternative to Aspera (2018)
131–140 of 172 posts
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#132Earlier quoted context omitted.
not true, although i'm sure there are one-off instances where there's no encryption. but in general there's lots of encryption on each of the different levels of secure government networks.
Right, they didn't want us encrypting what they were going to encrypt in their routers anyway. "Trusted" is a technical term. So they started using Aspera to send video from the UAVs over Iraq etc. to the Pentagon. Before that, they were literally flying boxes of videotapes (or CDs? Tapes seem hard to believe, except this was govt.) to a warehouse, where analysts could check them out, like from your neighborhood libr…
On first read I thought you meant Aspera ran on the UAVs :) but I now realize the high unlikeliness of that...
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#133Earlier quoted context omitted.
> The protocol totally ignores drops, for flow control. Instead, it measures change in transit time. The receiver knows, the sender needs to know, but the useful lifetime of the measurement is less than the transit time. This should make an engineer think "control theory!" (and did). So, the receiver reports a stream of transit time samples back to the sender, which feeds them into a predictor, which controls transmi…
I've reported most of what I know. I was involved in the data layer. Flow control was done by the PhDs. But, imagine the route as a series of packet queues. Packet is sent, arrives at the first router, spends time in the queue, gets sent, put on the next queue, waits, goes, until it's delivered. If the total time taken, compared to the last packet's, is more, that means the queues are growing. If the total time is le…
:)
How good is GNSS at simulating latency, buffer saturation, etc on a simulated network and how much complexity would I need to contemplate factoring in to build an accurate model of the average organically evolved, ad-hoc, multi-WAN-spanning sprawling corporate network?
I can't think of a more concrete, instructive starting point for someone interested in figuring out an alternative/reimplementation.
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#134There are other wan accelerators out there - like Riverbed. But, I think Aspera uses some specialized compression for genomics data, and genomics compression is not easy. My suggestion would be to talk to the telco that provides your wan. If you are using a direct internet connection my suggestion would be : don't.
The author is discussing intercontinental data transfers. Not many organizations have a WAN that can cross oceans without hitting the internet.
You have to buy access, but they do sell it.
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#135Earlier quoted context omitted.
almost all HE and research institutions participate in nrens which provide 3xacrly those peerings
Yes, but that typically takes government cooperation, and you still don't get better latency -- even if you have neutrinos.
Latency isn't the problem here - it's throughput.
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#136Earlier quoted context omitted.
what?
Certain labs were sending neutrinos right through the earth, gaining a measurable reduction in transit time vs great-circle overland signal propagation. (At first they thought they were gaining even more, by superluminal transit, but that turned out to illusory.) They did not put any data in the neutrino channel, to my knowledge, but in principle they could have. You might think you would need quite a deep chord to b…
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#137Ex Aspera Dev here. I did the encryption and early parallel work. There is a lot of good science behind fasp. An advantage it has over IETF protocols is that both ends trust one another. Another advantage, until recently, was out-of-order delivery. The protocol totally ignores drops, for flow control. Instead, it measures change in transit time. The receiver knows, the sender needs to know, but the useful lifetime of…
But out of order delivery and (I'm guessing) out of order retransmits appear to be unique to fasp. What did you mean about these being unique "until recently"? [edit - nevermind, saw below you were referring to sack]
PS. I remember looking at fasp ~10 years ago and it looked like a fantastic tech. Someone's taking things that many people only talked about and putting them into a product that really worked well. Cutting edge stuff. A job like - a dream for many :)
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#138[1]: https://en.wikipedia.org/wiki/UDP-based_Data_Transfer_Protoc...
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#139Earlier quoted context omitted.
The systems that produce massive DNA sequence files are machines reading the sequences of actual physical DNA molecules. The cloud can't sequence DNA, as far as I know.
Most DNA sequences are only slightly different from sequences you already have. There ought to be mature differs by now that would enable you to send just differences. If not, there is a huge opportunity for somebody.
https://en.wikipedia.org/wiki/Compression_of_Genomic_Sequenc...
Re: A Desperate Plea for a Free Software Alternative to Aspera (2018)
#140Earlier quoted context omitted.
>"Another advantage, until recently, was out-of-order delivery." Can you elaborate on this? Which recent development are you referring to? Thanks.
Doesn't TCP have an ability to re-send dropped packets, now, without sending everything after them? I thought they called it selective re-send, or selective ack, or something. Maybe the receiver could send an ACK with a bitmap of which packets it got, and the sender would fill in the holes?