Probability distributions are approximate world models.
There are mathematical tools to management them.
Once you see the world this way, ML/DL becomes more intuitive.
Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
101–110 of 123 posts
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#102Good slides, got me back in to the fever of wanting to learn; although a lot of the credit goes to the linked 3Blue1Brown videos (whose Linear Calculus series is excellent) which were a lot more technical but no less approachable. Question to those versed in ML: I want to work on an AI that plays a video game (aspirations of playing something like Rocket League, but I know I need to start smaller with something like…
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#103Earlier quoted context omitted.
I both make most of my money from time series data and use deep learning and work with data with no labels. Here's a recent presentation I did on some of this work and a companion presentation I encourage people to read on how to use this effectively in production. While you are right that some feature engineering is needed, there's no reason DL can't be a part of your workflow. https://www.slideshare.net/agibsonccc/…
> While you are right that some feature engineering is needed, there's no reason DL can't be a part of your workflow. I (and, I believe, the earlier poster, too) never implied you can't use deep learning on such examples. What we (I think, both) were referring to was the claim that it would absolve you from feature engineering. (Which I understand you also refute.) > For more of the basics, my book on deep learning m…
A lot of people make money with deep learning with images ;).
I guess what I wanted to do was add a bit of nuance. It can help reduce the amount of feature engineering needed. Of course you still need a baseline representation though. More feature engineering also doesn't hurt. I always think of deep learning in the time series context as a neat SVM kernel with some compression built in. With the right tuning it can give you a better representation which you can use with clustering and whatever else you'd like.
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#104As someone who works with a lot of people new to machine learning, I appreciate guides like this. I especially like the early slides that help frame AI vs ML vs DL so that people can have a realistic understanding of what these technologies are for. For my part, one of the biggest realization I had after many years of applying machine learning was that I got too caught up in the machine learning algorithms themselves…
What are best resources for "defining and generating" labels? Any recommendations?
There were some folks in this area targeting generating annotation tools of various kinds. I believe the focus was on something reducible to a web interface but if you can label things going that route check it out https://en.wikipedia.org/wiki/CrowdFlower
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#105The document is awesome, but the animated backgrounds are distracting.
I tried to edit it to prevent animation and apparently I can't edit or copy or download or export. Gotta try printing next.
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#106Information is great, but it would be much more readable in simple text form or pdf. It's strange that senior creative engineer at Google doesn't know presentation making basics.
It's not surprising that a Google engineer would use Google docs. It's at least easily shareable and there are complementary embedded videos that aren't suitable for text/PDF anyway. Though, the options to export as a PDF didn't work for me (either via download or as an export to Google Drive). I'm assuming the presentation is too big.
It has been explicitly set to be non-downloadable or exportable.
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#107I really appreciate you sharing this, as myself always try to create a learning path and make things clearer as much as possible on top of it, slides are good looking (design awesome). the problem sometimes comes is at its overwhelming when advanced things pop up in talk and its hard to know what should we listen to it or is it above our level of the current situation.
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#108Earlier quoted context omitted.
AI is the overall field and the superset of all of the various approaches. ML is one family of approaches for knowledge acquisition in AI, but far from the only one (eg. logic based inference is another big one). DL is a family of approaches in supervised ML. As the author points out, it's a subset of a subset. But saying that this sub-subset "drives" AI is like saying endocrinology "drives" medicine: not the right m…
If endocrinology was the most talked-about, hyped, and invested-in form of medicine I think it would be fair. DL is where an enormous amount of AI growth and progress is.
hyped things are made hype from a belief that it can be 'done' therefore invested in, but that can be done in one part of ML, which doesn't give any reason to say it generates all intelligence for a machine for human tasks.
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#109Earlier quoted context omitted.
One very old tool for such things was called "stepwise regression". IIRC J. Tukey was partially involved in that. It appears that the AI/ML work is close to the regression and curve fitting going back strongly to the early days of computers in the 1960s and a lot in the social sciences back to the 1940s and even about 1900. A lot is known. E.g., there's the now classic Draper and Smith, Applied Regression Analysis .…
Stepwise regression is not to be recommended because it's very easy to fool oneself. http://www.sascommunity.org/mwiki/images/e/e2/NYASUG-2007-Ju... http://www.barryquinn.com/the-statistical-dangerous-of-stepw... Shrinkage methods like lasso/elasticnet are less susceptible to these problems.
I agree fully. No doubt Breiman took some steps beyond what Tukey did. My only point was that the question and some answers are old.
Re: Machine Learning 101 slidedeck: 2 years of headbanging, so you don't have to
#110Earlier quoted context omitted.
One very old tool for such things was called "stepwise regression". IIRC J. Tukey was partially involved in that. It appears that the AI/ML work is close to the regression and curve fitting going back strongly to the early days of computers in the 1960s and a lot in the social sciences back to the 1940s and even about 1900. A lot is known. E.g., there's the now classic Draper and Smith, Applied Regression Analysis .…
Thank you for the list of resources. Are you able to go into more detail about your startup (problems it is solving)?
Okay, here, just for you. Don't tell
anyone!
My view is that currently there is a lot of content on the Internet and the total is growing quickly. So, there is a need -- people finding what content they will like for each of their interests.
My view is that current means for this need do well on (rough ballpark guesstimate) about 1/3rd of the content, searches people want to do, and results they want to find. My work is for the "safe for work" parts of the other 2/3rds.
The user interface is really simple; the user experience should be fun, engaging, and rewarding. The user interface, data used, etc. are all very different from anything else I know about.
The crucial, enabling core of the work, the "how to do that", the "secret sauce", is some applied math I derived. It's fair to say that I used some advanced pure math prerequisites.
To the users, my solution is just a Web site. I wrote the code in Microsoft's Visual Basic .NET 4.0 using ASP.NET for the Web pages and ADO.NET for the use of SQL Server.
The monetization is just from ads, at first with relatively good user demographics and later with my own ad targeting math.
The Web pages are elementary HTML and CSS. I wrote no JavaScript, but Microsoft's ASP.NET wrote a little for me, maybe for some cursor positioning or some such.
The Web pages should look fine on anything from a smart phone to a high end work station. The pages should be usable in a window as narrow as 300 pixels. For smaller screens, the pages have both horizontal and vertical scroll bars. The layout is simple, just from HTML tables and with no DIV elements. The fonts are comparatively large. The contrast is high. There are no icons, pull-downs, pop-ups, roll-overs, overlays, etc. Only simple HTML links and controls are used.
Users don't log in. There is no use of cookies. Users are essentially anonymous and have some of the best privacy. For the user to enable JavaScript in their Web browser is optional; the site works fine without JavaScript -- without JavaScript maybe sometimes users will have to use their pointing device to position the cursor.
There is some code for off-line "batch" processing of some of the data. The code for the on-line work is about 24,000 programming language statements in about 100,000 lines of typing. I typed in all the code with just my favorite text editor KEdit.
There is a little C code, and otherwise all the code is in Microsoft's Visual Basic .NET. This is not the old Visual Basic 6 or some such (which I never used) and, instead, is the newer Visual Basic part of .NET. This newer version appears to be just a particular flavor of syntactic sugar and otherwise as good a way as any to use the .NET classes and the common language runtime (CLR), that is, essentially equivalent to C#.
The code appears to run as intended. The code should have more testing, but so far I know of no bugs. I intend alpha testing soon and then a lot of beta testing announced on Hacker News, AVC.COM, and Twitter.
For the server farm architecture, there is a Web server, a Web session state server, SQL Server, and two servers for the core applied math and search.
I wrote the session state server using just TCP/IP socket communications sending and receiving byte arrays containing serialized object instances. The core work of the Web session state server is from two instances of a standard Microsoft .NET collection class, hopefully based on AVL or red-black balanced binary trees or something equally good.
The Web servers do not have user affinity: That is, when a user does an HTTP POST back to the server farm, any of many parallel Web servers can receive and process the POST. So, the Web servers are easily scalable. IIRC, Cisco has a box that will do load leveling of such parallel Web servers. Of course, with the Windows software stack, the Web servers use Microsoft's Internet Information Server (IIS). Then IIS starts and runs my Visual Basic .NET code.
Of course the reason for this lack of user affinity and easy scalability is the session state server I wrote. For easy scalability, it would be easy to run hundreds of such servers in parallel.
I have a few code changes in mind. One of them is to replace the Windows facilities for system logs with my own log server. For that, I'll just start with my code for the session state server and essentially just replace the use of the collection class instances with a simple file write statement.
I wrote no prototype code. I wrote no code intended as only for a "minimum viable product". So far I see no need to refactor the code.
The code is awash in internal comments. For more comments, some long and deep, external to the code, often there are tree names in the code to the external comments, and then one keystroke with my favorite editor displays the external comments. I have about 6000 files of Windows documentation, mostly from MSDN, and most of the tree names in the comments are to the HTML files of that documentation.
I have a little macro that inserts time-date stamp comments in the code, e.g.,
Modified at 23:19:07 on Thursday, December 14th, 2017.
and I have some simple editor macros that let those comment lines serve as keys in cross references. That helps.
The code I have is intended for production up to maybe 20 users a second.
For another factor of 10 or 20, there will have to be some tweaks in some parts of the code for more scaling, but some of that scaling functionality is in the code now.
For some of the data, a solid state drive (SSD), written maybe once a week and otherwise essentially read-only, many thousands of times a day, would do wonders for users served per second. Several of the recent 14 TB SSDs could be the core hardware for a significant business.
Current work is sad -- system management mud wrestling with apparently an unstable motherboard. At some effort, I finally got what appears to be a good backup of all the files to an external hard disk with a USB interface. So, that work is safe.
Now I'm about to plug together another computer for the rest of the development, gathering data, etc.
I'm thinking of a last generation approach, AMD FX series processor, DDR3 ECC main memory, SATA hard disks, USB ports for DVD, etc., Windows 7 Professional 64 bit, Windows Server, IIS, and SQL Server.