Machine learning == statistics
Deep learning == machine learning
41–50 of 78 posts
Machine learning == statistics
Deep learning == machine learning
Earlier quoted context omitted.
> data ownership and pipelines not ML. By your argument, the money is in data hoarding and brokering, and renting that data (with DRM) to ML outfits, not doing ML. Anyway, cleaning dirty data isn't execeptionally hard, it's just boring work; getting the raw data is the hard part.
Um, yes and no. Some of the data annotation problems get really hard and really expensive. Look at radiology or genomics. You literally need N people to die for more knowledge in some of these situations.
Because there is a real moat with data ownership and pipelines. If you want to do any analysis you quickly find that learning to properly use scikit-learn and tensorflow (or your machine learning algorithm of your choice) is atleast an order of magnitude lower of work than getting the data. For instance, I wanted to build a machine learning algorithm which took simple data from the SEC filed 10-Q and 10-K, which are…
> data ownership and pipelines not ML. By your argument, the money is in data hoarding and brokering, and renting that data (with DRM) to ML outfits, not doing ML. Anyway, cleaning dirty data isn't execeptionally hard, it's just boring work; getting the raw data is the hard part.
I think in practice, what happens is that there's no easy transactional boundary that can keep the data ownership and ML in separate firms. The theory of the firm [1] states that firms arise when transaction costs are less than the economic inefficiencies of centralized resource allocation. There are some pretty heavy transaction costs between doing data cleaning and data science in separate organizations:
1.) DRM for datasets isn't really a thing, since to explore, visualize, and train on them, you need access to the raw data, and then instead of your machine-learning function you can just pass the identity function to get the raw data. DRM for consumers always relies on a publisher whose incentive is to stay in business (by not breaking any laws) rather than to obtain the raw data.
2.) You don't know whether a cleaned data set will be useful for your ML application until you've inspected it, visualized it, run some statistics over it, etc. at which point you've done most of the work for setting up your models. That means there's a big risk premium for buying data, and potential buyers usually want samples & statistics before committing a lot of money.
3.) At the price that good datasets go for, you're usually dealing with enterprise sales, which involves commissioned salespeople, face-to-face meetings, expensive dinners out, etc.
4.) The type of labeling you need to do is often intimately connected with the usage of the ML model. What constitutes spam? What constitutes abuse? These are questions for your policy team, which the data-collection organization would have no visibility into and no way to set up a one-size-fits-all policy that all potential customers would be okay with.
5.) Software markets tend to be winner-take-all, which means you're dealing with a handful of customers, and one will likely become dominant and able to acquire you.
Instead of separate firms for data ownership, ML, and user interface, usually the service that makes the final product will end up collecting or buying the data outright, hire data-scientists to do the ML, and then offer a product. The end result, economically, is what you say: the money is in data hoarding. But it isn't apparent in prices of a sustainable customer ecosystem, it's apparent in acquisition prices for startups with data vs. salaries of data scientists.
There are surely some startups for which this is bullshit. But the good version of it is: - take some valuable task that's never been successfully automated before - do it manually (and expensively) for a while to acquire data - build an automated system with some combination of regular software and ML models trained on the data - now you can do a valuable task for free - scale up and profit The risk is that it's har…
Earlier quoted context omitted.
I can't think of a company that wouldn't have valuable data. A core use of ML is reducing costs by making better spending decisions/reducing waste and that is relevant to almost every company I think.
Recurring expenses (i.e. the kind that will generate enough data to adequately train a ML system) are already handled pretty well by most businesses through traditional (non-ML) methods. Operations Research has been a thing for 60+ years. Businesses that don't handle their recurring expenses well now are likely to have organizational issues (e.g. senior management that ignores the advice of their reports on how to do…
Because there is a real moat with data ownership and pipelines. If you want to do any analysis you quickly find that learning to properly use scikit-learn and tensorflow (or your machine learning algorithm of your choice) is atleast an order of magnitude lower of work than getting the data. For instance, I wanted to build a machine learning algorithm which took simple data from the SEC filed 10-Q and 10-K, which are…
This is the most basic securities data that exists in finance, perhaps you can even get it for free from some Yahoo or other retail data source.
Consider this: a good deal of open datasets are stolen. People had their faces taken without consent. The data was tagged and labelled for pennies per image on some server farm. Wrapping it up in machine learning can effectively black-box what it is you're actually doing.