The github repo has 5 million lines of C++ code (headers included), 1.6 million lines of C code, and even nearly 1 million lines of Scala + Java code. We'd need some serious docs to adopt this technology. The most interesting part of YT is Cypress. I'm particularly interested in how they make their master cluster horizontally scalable.
I was going over some of the code in the core folder for concurrency, threading and compression, what surprised me is that there’s absolutely no comments whatsoever. Agree that unless there’s excellent documentation, open source maintenance might be challenging.
Having said that, this definitely does look to be an impressive feat of engineering!
This looks very impressive! As another commenter echoed, the code base is ~5million lines of C++ code, but almost no comments at all. Unless the documentation is excellent, maintenance/open source work is going to be difficult.
P.S. I wonder if LLMs could be used to generate docs and comments for big hairy codebases. Seems that the current generation of LLMs lack context to do it, but maybe it's "just one or two more papers down the line"®...
There is an interesting take that Russian part of the Yandex group opensources as much as possible, in order for the overseas companies of the group to leverage technologies without legal or financial ties with Russia. For me this seems very plausible, as for the last year they first did everything to distance from anything related to politics (e. g. they sold their news and their blogging platform to the basically s…
russian Yandex no longer belongs to original Yandex founders/owners - it belongs to and is controlled by kremlin. makes sense that original engineers/founders create their own stuff via opensourcing their original work
Then why original Yandex founder is under EU sanctions?
Spoiler: because until June 22 (effectively until the sanctions hit) he was a Yandex CEO and owner of 8% shares (45% voting shares).
There is no "almighty Kremlin" that owns everything. There is, however, a set of rules you must comply to if you want to do multibillion dollar business in Russia. You either bend, or sell your business to more complacent oligarchs. Durov chose the latter, Volozh chose the first.
It's a problem of entire Russia. My Asian girlfriend from china was denied to enter a restaurant in Moscow because of her origins... It's absolutely legal to put restrict your rental apartments nationality. In yandex you can ads like " will lease my house only to russians". It's everywhere. If you will ride a taxi in russia, an offensive words to other nationalities/ethnicities is everywhere. The problem is so big, t…
> My Asian girlfriend from china was denied to enter a restaurant in Moscow because of her origins... While I'm not questioning your statement - this sound very questionable, considering the number of Chinese tourists in Moscow every year. >It's absolutely legal to put restrict your rental apartments nationality >I personally trying not to invest into anything Russian, simply because that society is very, very sick D…
Hello! I work at YT and would like to answer a question that was asked in a flagged thread about the comparison between YT and Hive and Zookeeper. Both Cypress and Zookeeper are fault-tolerant distributed hierarchical filesystems that can be used for distributed coordination, but Cypress has much richer functionality. Recall that Zookeeper's data model is just a tree consisting of homogeneous nodes that can be either…
I wonder if this thing is some kind of descendant of Elliptics used in Yandex internally a decade ago.
This looks very impressive! As another commenter echoed, the code base is ~5million lines of C++ code, but almost no comments at all. Unless the documentation is excellent, maintenance/open source work is going to be difficult.
The docs, for the reference: https://ytsaurus.tech/docs/en/ P.S. I wonder if LLMs could be used to generate docs and comments for big hairy codebases. Seems that the current generation of LLMs lack context to do it, but maybe it's "just one or two more papers down the line"®...
The cost of wrong docs is pretty high. You’d need someone knowledgeable to make corrections
I work at Cloudera and I see it as roughly analogous to CDP, but I didn't see anything about hybrid on-prem/cloud deployments.
It's true. We are very close to CDP and Apache Hadoop. We are only about on-prem right now. It's not a secret that we didn't need a cloud deployment at Yandex. But we see and understand the demand for it. And we have a plan:)
Interesting! I'm looking forward to seeing what you have lined up.
> It's illegal but the rental contract is private, so it can be denied without explanation; good luck suing and proving that you're being discriminated. How is this different than in other places in Europe
When worded like this, it's not that different. It's the little details that add up, depending on the actual country. You're much less likely to be discriminated against in UK or Germany than in places like Bulgaria, Ukraine, or Russia. Due to both the attitude and enforcement. The rental market in Germany seems over-regulated, but my black friend of Ethiopian descent (he's Russian, born and raised) had no problem fi…
i wonder if blatant discrimination against black people is less likely to occur in places like UK, Germany, France, Netherlands, Belgium, Australia, Canada, and US (ok not really the US but lets say for sake of argument) because of the collective guilt white people in those countries carry for colonization, slavery, and other shitty things; whereas slavic and other eastern european people never really did any of those really shitty things to black people. not only that but if i remember correctly ussr and early china provided quite a bit of aid to third world countries in order to overthrow collonialism. that said any discrimination against people is plain disgusting
brainwashing at its peak. When USA does things, it is for the good of society, democracy. When Russia does things, it is hurting people, bad for society. Come on buddy, time to wake up and understand every country does things for its own good and whatever your media is telling about Russia is bad, it is because they're applying 3 letter agency brainwashing methods on to you. Code is open source, if you read code you…
Would you remind when US committed genocide, mass execution of POWs, execution and torture of civilians, mass rape of women and children, and absolutely widespread looting in occupied towns?
I'm waiting.
I've read the whole article, it's very long, but it was written in such a refreshingly clean way. Hats-up!
I believe the engineers really spent a lot of their time building this from the ground up. I'm thankful that those large companies open-source their software and algorithms.