Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Algorithm Leaked

seroundtable.com

Post reply on HN
Print view View original

1–10 of 12 posts

Re: Google Algorithm Leaked

#2

Post by advisedwang »

It's not clear to me whether the leak is actually for Google Search or one of the products around search that isn't "Search", like Document Warehouse [1]. Is there anything definitive one way or the other in all this? Nobody seems to even questioning this

[1] https://cloud.google.com/document-warehouse/docs/overview

Re: Google Algorithm Leaked

#3

Post by atonse »

This looks like it's written in Elixir (the docs are using ExDocs, Elixir's documentation toolset).

This can't possibly be the actual search index rules (which is probably code that's decades old, my guess is either in Python or Java?) – unless they rewrote all of it in the past few years?

Can anyone else confirm this?

Re: Google Algorithm Leaked

#4

Post by 9dev »

atonse wrote: post #3

This looks like it's written in Elixir (the docs are using ExDocs, Elixir's documentation toolset). This can't possibly be the actual search index rules (which is probably code that's decades old, my guess is either in Python or Java?) – unless they rewrote all of it in the past few years? Can anyone else confirm this?

It’s not. Google uses a content warehouse database internally that holds all stored web page content, and to access this vast database, they have an API. The code discovered here is a generated SDK for Elixir for this content warehouse API.

Apparently, Google had a now deprecated product (who would have guessed that? Consider me shocked!) that provided customers with a trimmed-down version of this database for their own purposes, but mistakenly published the internal SDK code instead of that intended for Google Cloud customers to GitHub.

So while this doesn’t directly show the search index source code, it describes the data schema of the index in great detail, so there are at least some interesting educated guesses on the workings of the actual index to draw from it.

Re: Google Algorithm Leaked

#6

Post by 9dev »

advisedwang wrote: post #2

It's not clear to me whether the leak is actually for Google Search or one of the products around search that isn't "Search", like Document Warehouse [1]. Is there anything definitive one way or the other in all this? Nobody seems to even questioning this [1] https://cloud.google.com/document-warehouse/docs/overview

If you read the original publication on this[1], they mention there’s a stray commit publishing the internal variant of the SDK intended for the actual Google warehouse database. So the code bases probably live close enough together for someone to accidentally pass the wrong folder name or something.

This has been fixed, but the commit and all it’s changes are out there—and tragically, published alongside a copy of the Apache 2.0 license (intended for the document warehouse API SDK), which officially sanctioned freely copying and using the code. So there is really nothing Google can do about it.

[1] https://ipullrank.com/google-algo-leak

Re: Google Algorithm Leaked

#7

Post by lambdaxyzw »

9dev wrote: post #6
advisedwang wrote: post #2

It's not clear to me whether the leak is actually for Google Search or one of the products around search that isn't "Search", like Document Warehouse [1]. Is there anything definitive one way or the other in all this? Nobody seems to even questioning this [1] https://cloud.google.com/document-warehouse/docs/overview

If you read the original publication on this[1], they mention there’s a stray commit publishing the internal variant of the SDK intended for the actual Google warehouse database. So the code bases probably live close enough together for someone to accidentally pass the wrong folder name or something. This has been fixed, but the commit and all it’s changes are out there—and tragically, published alongside a copy of t…

>published alongside a copy of the Apache 2.0 license (intended for the document warehouse API SDK), which officially sanctioned freely copying and using the code. So there is really nothing Google can do about it.

This sounds good to us, tech nerds, but I'm pretty sure law doesn't work like this.

At minimum, even if (clearly accidentally) putting a code next to a open source license world be legally binding, the person doing the leaking was not the intellectual property owner. Google will just - correctly - say, that the person that leaked the code had no right to do so, especially under free license.

Re: Google Algorithm Leaked

#8

Post by natpalmer1776 »

lambdaxyzw wrote: post #7
9dev wrote: post #6

Earlier quoted context omitted.

If you read the original publication on this[1], they mention there’s a stray commit publishing the internal variant of the SDK intended for the actual Google warehouse database. So the code bases probably live close enough together for someone to accidentally pass the wrong folder name or something. This has been fixed, but the commit and all it’s changes are out there—and tragically, published alongside a copy of t…

>published alongside a copy of the Apache 2.0 license (intended for the document warehouse API SDK), which officially sanctioned freely copying and using the code. So there is really nothing Google can do about it. This sounds good to us, tech nerds, but I'm pretty sure law doesn't work like this. At minimum, even if (clearly accidentally) putting a code next to a open source license world be legally binding, the per…

To continue this train of thought, what is stopping a company from saying "This team did not ask for permission before they published XYZ under this license" and just rescinding access to a widely used library?

At what point can you no longer say "oopsie" ?

Re: Google Algorithm Leaked

#9

Post by [deleted] »

9dev wrote: post #6
advisedwang wrote: post #2

It's not clear to me whether the leak is actually for Google Search or one of the products around search that isn't "Search", like Document Warehouse [1]. Is there anything definitive one way or the other in all this? Nobody seems to even questioning this [1] https://cloud.google.com/document-warehouse/docs/overview

If you read the original publication on this[1], they mention there’s a stray commit publishing the internal variant of the SDK intended for the actual Google warehouse database. So the code bases probably live close enough together for someone to accidentally pass the wrong folder name or something. This has been fixed, but the commit and all it’s changes are out there—and tragically, published alongside a copy of t…

[deleted]

Re: Google Algorithm Leaked

#10

Post by bqmjjx0kac »

natpalmer1776 wrote: post #8
lambdaxyzw wrote: post #7

Earlier quoted context omitted.

>published alongside a copy of the Apache 2.0 license (intended for the document warehouse API SDK), which officially sanctioned freely copying and using the code. So there is really nothing Google can do about it. This sounds good to us, tech nerds, but I'm pretty sure law doesn't work like this. At minimum, even if (clearly accidentally) putting a code next to a open source license world be legally binding, the per…

To continue this train of thought, what is stopping a company from saying "This team did not ask for permission before they published XYZ under this license" and just rescinding access to a widely used library? At what point can you no longer say "oopsie" ?

The law is JIT. We don't need an answer now, just ask a judge when it happens.
Post reply on HN