Faster CRDTs: An Adventure in Optimization • Hacker News Board

Re: Faster CRDTs: An Adventure in Optimization

#91

Post by zladuric » Sat, Jul 31, 2021, 9:11 PM UTC

JW_00000 wrote: post #81

By the way, as someone who has published academic papers, if you're ever bothered about a paper or have some comments, don't hesitate to mail the authors. (Their e-mail addresses are always on the paper; especially target the first author because they have normally done the work.) We are happy to hear when someone has read our work and I at least would've liked to have known if someone found a problem with my papers.

> especially target the first author because they have normally done the work

As someone living with a recently promoted? (is that the correct term?) PhD in social sciences, this surprises me. Is that something specific for my country, for social sciences or my wife simply landed in a case full of rotten apples?

Re: Faster CRDTs: An Adventure in Optimization

#92

Post by kaba0 » Sat, Jul 31, 2021, 9:12 PM UTC

knuthsat wrote: post #14

vanderZwan wrote: post #7

On a meta-level, does anyone else think that the whole idea of writing a peer reviewed paper that is just a benchmark of different algorithms should be really rigorously reviewed before being accepted? Writing good benchmarks is hard, and so highly contextual that writing fair comparisons beteen algorithms (or data structures) is almost impossible unless you're an expert in all of the algorithms involved.

Problem is that academics are rarely experts at programming or have knowledge of computer architectures as much as someone in the industry. There are various tricks that are never taught at college, therefore academics have no idea some stuff even exists. Best example is discrete optimization research (traveling salesman, vehicle routing and its variants, schedule rostering etc.). Stuff you find in the papers there a…

You do realize there is a whole area of the research field dedicated for heuristic algorithms? They have a proper academic basis just as much as the “correct” solutions.

Re: Faster CRDTs: An Adventure in Optimization

#93

Post by lostdog » Sat, Jul 31, 2021, 9:13 PM UTC

vanderZwan wrote: post #7

On a meta-level, does anyone else think that the whole idea of writing a peer reviewed paper that is just a benchmark of different algorithms should be really rigorously reviewed before being accepted? Writing good benchmarks is hard, and so highly contextual that writing fair comparisons beteen algorithms (or data structures) is almost impossible unless you're an expert in all of the algorithms involved.

Benchmarking papers are inaccurate when the original algorithms are not open sourced, and the grad student needs to rewrite the algorithm from scratch. They can easily create different implementation details, and wind up with an algorithm that's slower than the original.

I do think that the original algorithm authors should have the opportunity to correct the benchmarking code, or to release their original implementation as open source to be benchmarked.

In some sense, the benchmarking paper with a slower implementation is more "correct," since an engineer evaluating which algorithm to use is just as likely to implement the algorithm in a slower way than the original paper. The incentives are right too: the original paper author should be giving enough details to recreate their work, and the benchmarker is showing that really their published algorithm is slow.

Re: Faster CRDTs: An Adventure in Optimization

#94

Post by benjismith » Sat, Jul 31, 2021, 9:19 PM UTC

josephg wrote: post #2

Hello HN! Post author here. I’m happy to answer questions & fix typos once morning rolls around here in Australia

I've been following your work for years (and I'm actually neck-deep in a ShareDB project right now) so I just want to say thank you for all of your contributions! I especially enjoyed this post.

Re: Faster CRDTs: An Adventure in Optimization

#95

Post by gfodor » Sat, Jul 31, 2021, 9:39 PM UTC

josephg wrote: post #2

Hello HN! Post author here. I’m happy to answer questions & fix typos once morning rolls around here in Australia

Thanks for ShareDB. It’s dope. I extended it to support collaborative voxel editing (https://jel.app) and works great.

Re: Faster CRDTs: An Adventure in Optimization

#96

Post by mesebrec » Sat, Jul 31, 2021, 9:43 PM UTC

zladuric wrote: post #91

JW_00000 wrote: post #81

By the way, as someone who has published academic papers, if you're ever bothered about a paper or have some comments, don't hesitate to mail the authors. (Their e-mail addresses are always on the paper; especially target the first author because they have normally done the work.) We are happy to hear when someone has read our work and I at least would've liked to have known if someone found a problem with my papers.

> especially target the first author because they have normally done the work As someone living with a recently promoted? (is that the correct term?) PhD in social sciences, this surprises me. Is that something specific for my country, for social sciences or my wife simply landed in a case full of rotten apples?

In my uni in Belgium, this is the case too. First author does most of the work.

Re: Faster CRDTs: An Adventure in Optimization

#97

Post by exmadscientist » Sat, Jul 31, 2021, 10:02 PM UTC

zladuric wrote: post #91

JW_00000 wrote: post #81

By the way, as someone who has published academic papers, if you're ever bothered about a paper or have some comments, don't hesitate to mail the authors. (Their e-mail addresses are always on the paper; especially target the first author because they have normally done the work.) We are happy to hear when someone has read our work and I at least would've liked to have known if someone found a problem with my papers.

> especially target the first author because they have normally done the work As someone living with a recently promoted? (is that the correct term?) PhD in social sciences, this surprises me. Is that something specific for my country, for social sciences or my wife simply landed in a case full of rotten apples?

At least in the fields I've published in, first author does the work (or at least most of it, almost always including writing the paper) and last author secured the funding. Sometimes authors are listed in alphabetical order, in which case one or two are usually listed as "corresponding author". The less-senior corresponding author usually did the work, and the more-senior corresponding author is usually either the one who secured the funding, or a long-term member of the research group who is likely to actually be around to answer correspondence in the future (and who also probably helped out enough to be worth corresponding with).

Re: Faster CRDTs: An Adventure in Optimization

#98

Post by anonymousDan » Sat, Jul 31, 2021, 10:04 PM UTC

zladuric wrote: post #91

JW_00000 wrote: post #81

By the way, as someone who has published academic papers, if you're ever bothered about a paper or have some comments, don't hesitate to mail the authors. (Their e-mail addresses are always on the paper; especially target the first author because they have normally done the work.) We are happy to hear when someone has read our work and I at least would've liked to have known if someone found a problem with my papers.

> especially target the first author because they have normally done the work As someone living with a recently promoted? (is that the correct term?) PhD in social sciences, this surprises me. Is that something specific for my country, for social sciences or my wife simply landed in a case full of rotten apples?

I think it's pretty field specific. In a lot of CS for example the first author did most of the work and the last author got the funding. The people in the middle may have done varying degrees of work ranging from helping substantially with the implementation, evaluation or writing of the paper to once having read the paper abstract :)

Re: Faster CRDTs: An Adventure in Optimization

#99

Post by robbles » Sat, Jul 31, 2021, 10:05 PM UTC

zladuric wrote: post #91

JW_00000 wrote: post #81

By the way, as someone who has published academic papers, if you're ever bothered about a paper or have some comments, don't hesitate to mail the authors. (Their e-mail addresses are always on the paper; especially target the first author because they have normally done the work.) We are happy to hear when someone has read our work and I at least would've liked to have known if someone found a problem with my papers.

> especially target the first author because they have normally done the work As someone living with a recently promoted? (is that the correct term?) PhD in social sciences, this surprises me. Is that something specific for my country, for social sciences or my wife simply landed in a case full of rotten apples?

I think the term you're looking for might be "postdoc" (postdoctoral researcher)? I've seen "recent PhD" a few times as well, seems pretty clear.

Re: Faster CRDTs: An Adventure in Optimization

#100

Post by lewisl9029 » Sat, Jul 31, 2021, 10:10 PM UTC

josephg wrote: post #63

Jailbird wrote: post #50

Earlier quoted context omitted.

Drifting off-topic but I've wondered this myself - I've been interested in CRDTs in a "read the news" way but not a "this rises to something I'm going to implement" way. Perhaps it's blindingly obvious to all here, so no one mentions it: Thinking about more practical and real-world problems seems like collaboration on on more complex/structured data. Today, it seems one would still have to transform that data to a la…

> Today, it seems one would still have to transform that data to a large flat string underneath, and implement an editor that only performs edits that maintain the integrity of the higher-level object, while the flat string provides collaboration features. Lots of people think this and have mentioned it over the years, but its a dangerous idea. The way concurrent edits are handled makes it really easy for the automat…

First of all, thank you for the amazing read! I thoroughly enjoyed the entire article, and it gave me a new perspective on the feasibility of CRDTs for real world applications performance-wise.

Though I am curious now to hear your thoughts on the conflict resolution side of the equation for complex data structures like deeply nested JSON.

The biggest takeaway I got from Martin's talk on the topic from a few years ago was that while CRDTs are theoretically guaranteed to eventually converge, the resulting converged state might not make any sense to applications that need to consume and act on that state [1].

It seemed like a pretty fundamental challenge to using CRDTs to store arbitrary application state to me at the time, but I imagine the field has progressed immensely since then, so would love to hear any insights you might have around strategies that could be used to alleviate or at least work around this challenge if I wanted to build a CRDT-based app today.

[1] https://youtu.be/8_DfwEpHE88?t=2232