Earlier quoted context omitted.
It's kind of remarkable. There really is no literature review in this paper. As a supervisor I would have no problem with a content part of this length, but I would also insist on doing the scholarly work that is not demonstrated here. Don't just throw out some code and describe it but put it into the context of what exists. Give credit to where ideas originate. That shouldn't add too much. No more than a few pages.…
From the readers perspective, what would be the point?
Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
81–90 of 102 posts
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#82An underappreciated aspect of this is finding an academic department that would allow you to submit something this concise as a senior thesis. My experience, mostly in grad school, was that anyone editing my work wanted more verbiage. If you only needed a short, one-sentence paragraph to say something, it just wasn’t accepted. There had to be more. Jeff Dean is an uncommonly good communicator. But he also benefited f…
I had to write a 10k word undergrad final paper for law school (in Europe, law school is a regular university study, with a 3 year LLB and 1 or 2 year LLM). At around 8k or so, I went to my supervisor and said 'look, I've said everything I wanted to say, and in a drawn out way already which I' not happy about. If I have to add more, I will need to start another topic, and I'd rather keep this paper focused and contin…
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#83As always, Jeff Dean doesn't fail to inspire respect. Tackling a complex problem (still relevant today) at an early age, getting great results and describing the solution clearly/concisely. My master thesis was ~60 pages long, and was probably about 1/1000 as useful as this one.
I put a lot of effort in my undergraduate thesis, but none of the professors on my committee had much interest in advising me; and after my defense, the only professor who really gave me his undivided attention came to me and said “I’m glad you’re not staying here for grad school; you’re way too good for this place”. ¯\_(ツ)_/¯ Comparing your journey to others’ is pointless.
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#84Earlier quoted context omitted.
> anyone editing my work wanted more verbiage I _never_ got that feedback. My mentors all emphasized economy of language and nobody cared how "thick" my thesis was. This is a pretty amusing story about verbiage. Back in the old days, you would send a manuscript/research article to colleagues/friends by _snail-mail_ to get their feedback. You'd wait a month, and maybe they would mail a 'red-inked' copy of your manuscr…
Of course people want you to write less when it means less work for them. But think of an academic review committee for a senior thesis. They aren’t going to heavily read your paper, just decide if it meets compliance standards for some preserved artifact for your graduation. But they can veto what you’ve written and send it back to someone else (your advisor likely) and have that person request edits. So this puts t…
I wrote an undergrad thesis and I felt like my advisor and two readers cared about what I was saying and how I said it.
Same is true for my Ph.D. My committee members seemed to care deeply about the work (I think they just didn't want to be associated w/ crap research).
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#85Earlier quoted context omitted.
Since you clearly like math, suppose there are 20 million students applying to college every year. One million of them are very bright (2-sigma intelligence). Suppose that every Ivy League accepts around two thousand students every year. Can the sum of Ivy Leagues fit all of the smart people? Bonus question: What if we change our constraints to only account for 3-sigma people? Does our conclusion change?
Of course there's a distribution of those folks at all other universities, especially at top public ivies like GTech and UCB. But the cumulative effect of being at a lower tier institution can be pretty significant when searching for future opportunities. One thing I've noticed among people at my school is that even if they are very capable, they often don't apply to top internships, top programs for grad school etc,…
This speaks to their lack of confidence more than their capability. Perhaps that is one of the advantages a good school, a good peer group, or a good network can confer: the confidence to aim higher.
People don't think they're good enough... which may be true, but no one can truly know until they try. Self-limiting thinking is particularly prevalent in rust-belt cities and regions where knowledge or achievement is not prized, so people in knowledge-intensive fields have no models to emulate.
And sometimes when they try (it's not unusual for graduates of lower ranked universities to send out 300+ resumes only to get single digit responses), they get demoralized when they don't succeed on their first few tries, when in fact there's more than one path in life -- if one doesn't have natural advantages, one might have to embrace the more circuitous path(s). This can mean joining a startup, going to a better grad school than one's undergrad, moving to a better city to upgrade one's peer groups (this is more important than most people think [1]), etc.
Life can surprise you if you keep trying and pivoting (ugh cliche, but there it is). There's an element of randomness and stochasticity in a free market, and I've seen enough counterexamples to distrust a static conception of how things "should be". (except for some stodgy areas like investment banking that only hire from certain schools; but even then there are backdoors)
* You're a new graduate, and the hardest hurdle you have to overcome is to get in the door. If you manage to do that and are able to prove yourself, your undergrad degree will become less and less and important. If you google Fortune 500 company CEOs, especially in non-tech companies, (you can do this exercise for yourself) you will learn that many of them went to non-elite schools for undergrad. For all its elite colleges, America is not really an academic-technocratic society (unlike countries like Germany where most CEOs have Ph.D.s). There are elements of William James' pragmatic philosophy that still influence the thinking in this country -- getting results is more important than academic knowledge.
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#86Earlier quoted context omitted.
Pretty interesting, reading his short bio I learned for the first time about AHPCRC ( https://ahpcrc.stanford.edu ) of which prof. Kumar was head of for about 7 years, the US military is indeed involved almost everywhere in SV.
Read Steve Blank's "The Secret History of Silicon Valley". The US military created Silicon Valley.
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#87Quite incredible that he was interested in NNs back in 1990. He closed this thread very well.
The interest in NNs was ignited (in part) by this double volume collection of essays called "Parallel Distributed Processing" edited by Rumelhart and McClelland.
Dean even cites them. And, if you read the contributors, it contains many (though not all) of the heavy hitters.
Reading back on it, it will sound very familiar. All the amazing breakthroughs: object recognition, handwriting recognition etc all seemed to be there. But all that rapid progress just seemed to stop. There was this quantum leap and then you were back to grinding out for even 0.1% improvement.
For those who stuck through the second winter, things obviously paid off.
The intro essay is online:
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#88My entire career has consisted of reimplementing bits and pieces of things I've previously built all the way back to high school and then reimplementing whatever was new on the previous round in the next one.
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#89Earlier quoted context omitted.
Thanks for the list but I am not entirely sure about Sergey Brin and Larry Page’s alma mater.
Interesting fact: those two went on to create the search engine Google. You should check it out.
Apologies for not making myself clear enough :(
Re: Google’s Jeff Dean’s undergrad senior thesis on neural networks (1990) [pdf]
#90Earlier quoted context omitted.
It's kind of remarkable. There really is no literature review in this paper. As a supervisor I would have no problem with a content part of this length, but I would also insist on doing the scholarly work that is not demonstrated here. Don't just throw out some code and describe it but put it into the context of what exists. Give credit to where ideas originate. That shouldn't add too much. No more than a few pages.…
That seems to be a key difference between science and engineering. One likes to survey the field and insist that their paper offers something novel - no matter how big or small. The other just wants to do some solid work and get the results.