Earlier quoted context omitted.
I’d be happy to chat more about your needs and try to help recommend a path forward. Feel free to shoot me an email at the address in my profile.
Is this an ad? Why can't the topic continue here as a reply to op?
TernFS – an exabyte scale, multi-region distributed filesystem
21–30 of 32 posts
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#22Earlier quoted context omitted.
Is this an ad? Why can't the topic continue here as a reply to op?
Because its a consulting oportunity.
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#23Earlier quoted context omitted.
Because its a consulting oportunity.
I read somewhere that Hacker News should have been named Startup News, and sometimes interactions like the one upthread reminds me of that. I'm not saying it's wrong - if you're good at something don't do it for free and all that - but it's kinda sad that in-depth discussions on public forums are getting harder and harder to find these days.
Agree about the public discussion part, one of the reasons why I'm here lately.
Also, why can't someone create Startup News: Where every article reply is an opportunity to be sold a service, SN would take a cut of transactions. /s
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#24To the OP: nice karma trick posting the URL with the anchor to bypass the HN duplicates detector. Dang & co, this is a bug, it should be fixed. I know because I stumbled on the same page following the links from the blog of the author of another post that made the frontpage yesterday ( https://news.ycombinator.com/item?id=45589156 ), liked the TernFS concept, submitted it and got redirected to https://news.ycombinato…
Agreed, more or less, this would be easy to work around naively. Though an duplicate detection should not block reposts based on removing the anchor nor should the anchor portion automatically be removed generally. Some sites unnecessarily setup as SPAs use the anchor portion for context so it is needed for direct links to the right article in those¹, and going directly to a specific section in a long page can also be useful.
> nice karma trick posting the URL with the anchor to bypass the HN duplicates detector.
Karma jealously is as unbecoming as karma whoring, so perhaps wind the ol' neck in a little there. Laziness/ineptitude is more common than malice and this could have been accidental via quick copy+paste.
A better way to respond in this situation is a more neutral “Already recently discussed at [link]”, as had been done some hours before your comment: https://news.ycombinator.com/item?id=45646691#45647047
----
[1] Yes, those sites are badly designed, but they are unlikely to change because of our technical preferences and breaking the ability to deep link into them would add an issue for HN while not being noticed by those sites at all.
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#25Earlier quoted context omitted.
I read somewhere that Hacker News should have been named Startup News, and sometimes interactions like the one upthread reminds me of that. I'm not saying it's wrong - if you're good at something don't do it for free and all that - but it's kinda sad that in-depth discussions on public forums are getting harder and harder to find these days.
Normal conversations by topic enthusiasts usually have fun stuff hidden in their profiles and at times lead to fun rabbit holes where you endlessly learn and somehow, forgot that you were initially browsing HN. Agree about the public discussion part, one of the reasons why I'm here lately. Also, why can't someone create Startup News: Where every article reply is an opportunity to be sold a service, SN would take a cu…
These are people already trying to divert the discussion off-site for their benefit. Very few would honestly report any resulting transaction for the cut to be taken from.
[yeah, it did see the sarcasm tag, just clarifying to put off would be entrepreneurs so we aren't inundated by show-hn posts from people vibe-coding the idea over the next few days!]
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#26To the OP: nice karma trick posting the URL with the anchor to bypass the HN duplicates detector. Dang & co, this is a bug, it should be fixed. I know because I stumbled on the same page following the links from the blog of the author of another post that made the frontpage yesterday ( https://news.ycombinator.com/item?id=45589156 ), liked the TernFS concept, submitted it and got redirected to https://news.ycombinato…
> Dang & co, this is a bug, it should be fixed. Agreed, more or less, this would be easy to work around naively. Though an duplicate detection should not block reposts based on removing the anchor nor should the anchor portion automatically be removed generally. Some sites unnecessarily setup as SPAs use the anchor portion for context so it is needed for direct links to the right article in those¹, and going directly…
Agreed, and sorry for that (even though my gut and not-so-gut feeling is that it was done on purpose rather than by mistake, but I might be on the wrong side myself on this).
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#27Earlier quoted context omitted.
> Dang & co, this is a bug, it should be fixed. Agreed, more or less, this would be easy to work around naively. Though an duplicate detection should not block reposts based on removing the anchor nor should the anchor portion automatically be removed generally. Some sites unnecessarily setup as SPAs use the anchor portion for context so it is needed for direct links to the right article in those¹, and going directly…
> Karma jealously is as unbecoming as karma whoring, so perhaps wind the ol' neck in a little there. Laziness/ineptitude is more common than malice and this could have been accidental via quick copy+paste. Agreed, and sorry for that (even though my gut and not-so-gut feeling is that it was done on purpose rather than by mistake, but I might be on the wrong side myself on this).
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#28To the OP: nice karma trick posting the URL with the anchor to bypass the HN duplicates detector. Dang & co, this is a bug, it should be fixed. I know because I stumbled on the same page following the links from the blog of the author of another post that made the frontpage yesterday ( https://news.ycombinator.com/item?id=45589156 ), liked the TernFS concept, submitted it and got redirected to https://news.ycombinato…
I think being able to identify what's worthy of reposting deserves upvotes, too. If a repost truly provided little to no value, then the number of upvotes would reflect that and it would never get to the front page. But in this case, many people and myself included would have never found the post if it weren't for this repost.
Different batches of users are on HN at different times and on different days. Allowing reposts to collect karma would mean that every link's exposure is derived from the entire HN userbase's votes rather than a small subset of the users that happened to be online at the time of the post.
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#29One of the pain points of scaling Zookeeper is that all writes must go to the leader (reads can be fulfilled by followers). I understand this is "leader of a shard" and not a "global leader," but it still means a skewed write load on a shard has to run through a single leader instance
> given that horizontal scaling of metadata requires no rebalancing
This means a skewed load cannot be addressed via horizontal scaling (provisioning additional shards). To their credit, they acknowledge this later in the (very well-written) article:
> This design decision has downsides: TernFS assumes that the load will be
> spread across the 256 logical shards naturally.
Re: TernFS – an exabyte scale, multi-region distributed filesystem
#30Earlier quoted context omitted.
There are several other systems I would recommend before TernFS for your environment. If you're looking at Lustre versus this in particular, Lustre has been through the wringer, and ANL/DOE has plenty of people who understand it enough to run it well and fix it when it breaks. However, you are right. Your bandwidth needs don't really require Lustre.
Seriously man, I'm asking because I don't know: which filesystems do you recommend instead? I dabbled in CephFS because our data is write-once, but helping computer illiterate research scientists at other universities and national labs retrieve their data is a lot simpler from Lustre because it's just plain-old POSIX filesystem semantics. I'm not joking, I didn't ask this as a way to namedrop my experience and creden…
Or just shell out for as much Weka as they can convince you that you need and call it a day.