Machine learning can boost the value of wind energy • Hacker News Board

Re: Machine learning can boost the value of wind energy

#31

Post by amrrs » Wed, Feb 27, 2019, 5:21 AM UTC

linkingday wrote: post #8

dmichulke wrote: post #5

I have been working in the same field, quite a few years in my own forecasting company and then later on as a side project. Not wanting to steal the thunder of Deepmind, but I feel should mention my site which is currently beta as in "works for a few electricity power load and supply forecasting scenarios" and quite well so. Also, it's out there, cheap and fast. https://ausblick.cryptoport.net Unfortunately, it's onl…

What prediction algorithm are you using for this? Not sure if it's mentioned elsewhere on the site

I have previously built such a tool for retail sales forecasting (based on previous sales). The stack used R + Rshiny (web app). Method FB's `prophet` as it seemed faster and more accounting of holidays and other variations.

Re: Machine learning can boost the value of wind energy

#32

Post by dmichulke » Wed, Feb 27, 2019, 5:34 AM UTC

linkingday wrote: post #8

dmichulke wrote: post #5

I have been working in the same field, quite a few years in my own forecasting company and then later on as a side project. Not wanting to steal the thunder of Deepmind, but I feel should mention my site which is currently beta as in "works for a few electricity power load and supply forecasting scenarios" and quite well so. Also, it's out there, cheap and fast. https://ausblick.cryptoport.net Unfortunately, it's onl…

What prediction algorithm are you using for this? Not sure if it's mentioned elsewhere on the site

No, it's not mentioned. It's currently a mix of very simple techniques

- a number of smart features (usually a few k) depending on the series (using lags, aggregates, curve fits, combinations of features, ...)

- an iterative algorithm that selects features using maximum relevance (~ correlation with the target) / minimum redundancy and adds them to the model

- simple pca and ridge regression (because it's fast)

- a few optimizations of the final model (removing features, selecting a better ridge regression alpha with CV, ...)

The stack is pure Clojure / Clojurescript.

Re: Machine learning can boost the value of wind energy

#33

Post by ychen306 » Wed, Feb 27, 2019, 6:09 AM UTC

Perhaps some people are missing the point here?. The post is not about the technical challenge/novel but rather how existing ML techniques can play a part in making reusable energy more efficient.

Re: Machine learning can boost the value of wind energy

#34

Post by joe_the_user » Wed, Feb 27, 2019, 7:01 AM UTC

sgillen wrote: post #26

tobias2014 wrote: post #25

How exactly does this need machine learning? Naively I would assume that it's somewhat directly correlated with wind speed (and little more, maybe direction) and a very simple model based on that data will give you good predictions.

I think determining the wind speed and direction 36 hours in advance is the tricky bit.

Weather forecasting is difficult problem but it's problem that as far as I know has been worked on intensely for quite a while.

Weather forecasting is based on physics-based models that, so far as I know, are essentially correct with the fundamental barrier to prediction being a combination of random effects and the difficulty of solving the complex equations numerically.

Deep learning involves training a heuristically approximate a system based on past data. It has been used to emulate various human-learning behaviors like recognizing images or recognizing good or bad position in games. Essentially, the visible successes are in "we don't know but can extrapolate roughly from data" (or we "know" but can't easily program it, in the case of image recognition). So I'd be surprised if deep learning do a good or better job in situation where we do know how things work, we already running algorithm which is correct - the situation with weather.

Re: Machine learning can boost the value of wind energy

#35

Post by qsolo » Wed, Feb 27, 2019, 7:13 AM UTC

I was super excited when I read the headline and saw that deepmind published the article. I was hoping for something along the lines of applying reinforcement learning on the turbine or park level to optimize power output. Finding the optimal pitch angles, better yawing algorithms (yaw misalignemnts are one of the most common causes of underperforming wind turbines), or finding optimal yaw directions for the entire wind park (downwind wake effects can have a negative impact on other turbines)...

Unfortunately this is really only about predicting wind speeds to then forecast power output. IIRC, grid codes in some countries are starting to require 24 hour, 15 minute interval forecasts for renewables? I've also seen this feature being advertised by other companies over the years as well, although I don't recall seeing any numbers on accuracy for any of the models.

Re: Machine learning can boost the value of wind energy

#36

Post by angry_octet » Wed, Feb 27, 2019, 10:30 AM UTC

"Statistics are useful for forecasting."

Meteorologists have long used a metric, forecast skill, to evaluate different techniques. It would be good if the researchers used standard benchmarks.

ML is probably much more useful in understanding the demand for power, the reaction to weather and current events. But even still, it seems a stretch.

Re: Machine learning can boost the value of wind energy

#37

Post by profunctor » Wed, Feb 27, 2019, 12:24 PM UTC

dmichulke wrote: post #32

linkingday wrote: post #8

Earlier quoted context omitted.

What prediction algorithm are you using for this? Not sure if it's mentioned elsewhere on the site

No, it's not mentioned. It's currently a mix of very simple techniques - a number of smart features (usually a few k) depending on the series (using lags, aggregates, curve fits, combinations of features, ...) - an iterative algorithm that selects features using maximum relevance (~ correlation with the target) / minimum redundancy and adds them to the model - simple pca and ridge regression (because it's fast) - a f…

How did you find developing your models in clojure? I would love to switch away from python for ml but it seems to just have the ecosystem.

Re: Machine learning can boost the value of wind energy

#38

Post by dmichulke » Wed, Feb 27, 2019, 7:17 PM UTC

profunctor wrote: post #37

dmichulke wrote: post #32

Earlier quoted context omitted.

No, it's not mentioned. It's currently a mix of very simple techniques - a number of smart features (usually a few k) depending on the series (using lags, aggregates, curve fits, combinations of features, ...) - an iterative algorithm that selects features using maximum relevance (~ correlation with the target) / minimum redundancy and adds them to the model - simple pca and ridge regression (because it's fast) - a f…

How did you find developing your models in clojure? I would love to switch away from python for ml but it seems to just have the ecosystem.

Actually, I use both but my production models are in Clojure.

I often end up implementing minor things myself using lower level abstractions (e.g., Linear Regressions or PCA with whitening using Matrix libraries) and I check the results and/or try new things using scikit-learn.

So in general, I'd say I do the programming (outputing intermediate CSVs, tests, web service, thread handling, UI, ...) in Clojure(Script), and try other approaches (e.g., other models/parameters/...) in Python.

I'm quite happy with this pipeline but probably to some extent because I really love to understand how things work and nothing pushes you to learn as much as a missing function in your ML library :-)