I thought the most common file names had some peculiar results. > the filename “15” & “s15” made the top 50 most popular filenames Anyone know why? https://boyter.org/posts/an-informal-survey-of-10-million-gi...
Processing 40TB of code from 10M projects with a dedicated server and Go
21–30 of 83 posts
Re: Processing 40TB of code from 10M projects with a dedicated server and Go
#22> If someone wants to host the raw files to allow others to download it let me know. It is a 83 GB tar.gz file which uncompressed is just over 1 TB in size. Some of the cloud providers have free hosting for public data sets (people who use the data incur cost to download/process the data). I'm not sure if this would qualify. * https://aws.amazon.com/opendata/public-datasets/ * https://azure.microsoft.com/en-us/servic…
Re: Processing 40TB of code from 10M projects with a dedicated server and Go
#23Re: Processing 40TB of code from 10M projects with a dedicated server and Go
#24> If someone wants to host the raw files to allow others to download it let me know. It is a 83 GB tar.gz file which uncompressed is just over 1 TB in size. Some of the cloud providers have free hosting for public data sets (people who use the data incur cost to download/process the data). I'm not sure if this would qualify. * https://aws.amazon.com/opendata/public-datasets/ * https://azure.microsoft.com/en-us/servic…
https://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPay...
Re: Processing 40TB of code from 10M projects with a dedicated server and Go
#25plug https://www.softwareheritage.org/
Re: Processing 40TB of code from 10M projects with a dedicated server and Go
#26Re: Processing 40TB of code from 10M projects with a dedicated server and Go
#27Re: Processing 40TB of code from 10M projects with a dedicated server and Go
#28This is a cool project and a great write-up. Some of the complexity numbers struck me as pretty off though. I know he caveats you can only compare files of the same language, but I took a quick look at the code[1] and at least as of now it looks like the complexity statements are just kind of copy/pasted, so keywords not seen in Java/C such as "match" are missing from Rust, OCaml, and Scala, "case" is missing from Ha…
Re: Processing 40TB of code from 10M projects with a dedicated server and Go
#29I thought the most common file names had some peculiar results. > the filename “15” & “s15” made the top 50 most popular filenames Anyone know why? https://boyter.org/posts/an-informal-survey-of-10-million-gi...
I would like to know to, any insights?
15 is also a common number in coding problem sets that people post to GitHub. It’s a stretch but it might be something to do with 2015 being a year that has a lot of coursework commits, and 15 is a pretty low number (i.e. higher probability a student posts solutions to #1-15 than #1-19).
The same goes for schoolwork. Many people do online MIT and Stanford courses that are from previous years ~2012-2016, but there has been less time elapsed for students to post answers from 2016,17,18,19)
This is mostly conjecture, so I hope someone has a better answer!