Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Antifragility in complex dynamical systems

nature.com

Post reply on HN
Print view View original

61–70 of 115 posts

Re: Antifragility in complex dynamical systems

#61

Post by baxtr »

sethammons wrote: post #46
_yb2s wrote: post #13

Earlier quoted context omitted.

Very cool, but I'm surprised you're sharing this on here and not in a job interview with a deep learning startup and/or an arXiv paper.

Your surprise breaks my brain. Why would someone "keep it to themselves" outside some white paper they won't have time to promote or for a job interview that they might not need? Is your thought that it is some novel approach that nobody is doing and has obvious market implications? Even then, the implications matter only if you are in a position to take advantage. The Internet used to be a place people shared cool s…

They are sharing it here?! It’s not like OP is keeping it to themselves

Re: Antifragility in complex dynamical systems

#62

Post by taylorius »

bob1029 wrote: post #2

I've found that intentionally causing abrupt (but reasonable) changes to hyperparameters in evolutionary spiking neural network simulations (I.e between each generation) results in far more robust simulations that will meet fitness criteria with less likelihood of getting stuck somewhere. The tradeoff being that simulating will take longer, but this may be worth things like reducing the chances of your resource requi…

Firstly, I'm coming at this from the pint of view of standard feed forward NNs, rather than spiking neural networks, so apologies in advance if I am way off-base.

By global activation limit, I assume you mean the total sum of neuron activation during one forward pass? So halving it would strongly encourage sparsity? How do you enforce the reduced limit? Just put it in the loss function? Or does the spiking nature allow you to "sparsify" the spiking by raising a threshold?

Very very interesting stuff. I'm working with differential evolution, and incorporate correlation measures between candidates to (try to) allow for multiple divergent solutions, and it also feels fairly off the beaten track, compared to mainstream AI. But who cares, let's enjoy the dark forest! :-)

Re: Antifragility in complex dynamical systems

#64

Post by brians »

lokimedes wrote: post #60

How odd that the word "Antifragility" has risen so fast. What's wrong with robust? Is it due to contemporary memes of personal fragility?

It’s trying to highlight a different idea. A robust system can endure damage. An antifragile system grows stronger from damage.

Re: Antifragility in complex dynamical systems

#65

Post by bob1029 »

phyalow wrote: post #41
bob1029 wrote: post #17

Earlier quoted context omitted.

> a job interview with a deep learning startup I am not aware of any DL startups that are interested in techniques which are incompatible with GPUs and back propagation. I think this is a solo journey through the dark forest for now.

Correct me if im wrong, this is just a question originating from my own curiosity, but could you not apply the same or similar techniques to classical NN architectures? I dont see exactly how the differentiability of the architecture matters? I.e. in between epochs randomly perturb weight values (similar effect I understand to introducing scheduled noise or non-isomorphic transforms on the features), implement a sche…

> I dont see exactly how the differentiability of the architecture matters?

I don't know that classical architectures are infeasible to perturb as noted in other comments. I just think we have a really interesting knob to turn in this specific case that doesn't seem to exist in other architectures.

How would you go about constraining the resource consumption (by up to half!) of a classical ANN while preserving realistic hopes that it will recover between training epochs? And, we are talking real resource reduction here not some academic thing. You can see the difference in task manager. That's actually the primary tool I've been using to determine how to tune the parameters of this scheme.

Re: Antifragility in complex dynamical systems

#66

Post by Eddy_Viscosity2 »

lokimedes wrote: post #60

How odd that the word "Antifragility" has risen so fast. What's wrong with robust? Is it due to contemporary memes of personal fragility?

I, along with several other commenters I see, also dislike "antifragility" as a word. However, Taleb in his book does discusses at length why a new word was needed and how its definition differs from existing words like 'robust', 'resilient', etc. which have a different meaning. He even talks even the process of choosing "antifragility" which in a nutshell was just the best he could come up with.

Seeing other comments, I think 'hormesis' might be a better word to start using instead of "antifragility" which, again, is a just an awkward word that feels kind of fake when you use it.

Re: Antifragility in complex dynamical systems

#67

Post by hobs »

BriggyDwiggs42 wrote: post #38
fsckboy wrote: post #11

Earlier quoted context omitted.

> but death isn't a system death is entropy. do you know of a system without entropy? > or organism... boy are you going to be surprised when you answer that knock on the door, eh?

Isn’t it definitely not an organism? If death is the inevitable process from order to disorder, isn’t it deeply a-systemic at its core? If it were a system, it would also eventually have to end.

Arguably it does, the heat death of the universe option would spell the end of entropy, and the end of death (unless you believe in a conformal cyclic universe)

Re: Antifragility in complex dynamical systems

#68

Post by seanhunter »

bob1029 wrote: post #2

I've found that intentionally causing abrupt (but reasonable) changes to hyperparameters in evolutionary spiking neural network simulations (I.e between each generation) results in far more robust simulations that will meet fitness criteria with less likelihood of getting stuck somewhere. The tradeoff being that simulating will take longer, but this may be worth things like reducing the chances of your resource requi…

> I've found that intentionally causing abrupt (but reasonable) changes to hyperparameters in evolutionary spiking neural network simulations (I.e between each generation) results in far more robust simulations that will meet fitness criteria with less likelihood of getting stuck somewhere.

If I understand you correctly this is similar to the philosophy behind simulated annealing isn't it? The idea being that adding some peturbation can be enough to jog the evolution out of local minima and you probably want to reduce the amount of those jogs over time so it ends up converging.

https://en.wikipedia.org/wiki/Simulated_annealing

Re: Antifragility in complex dynamical systems

#69

Post by AgentOrange1234 »

tokai wrote: post #49
AgentOrange1234 wrote: post #5

I wish folks would write more plainly. “Antifragility characterizes the benefit of a dynamical system derived from the variability in environmental perturbations.” Geish.

That sentence is completely fine, and not even close to being difficult. If that is not plainly enough for you, scholarly communication is not going to be possible.

I’ve written academic papers.

I’ve worked with amazing folks who aren’t native English speakers.

I’ve come to prize simple writing.

Re: Antifragility in complex dynamical systems

#70

Post by morning-coffee »

delichon wrote: post #4

Systems or organisms can be defined as antifragile if they derive benefit from systemic variability, volatility, randomness, or disorder. If that's a riddle, death fits. Those things are characteristic of the pestilence, famine, war, etc. that feed death. The decay surrounding death is complex and dynamical. And death is more sustainable than any living system.

What's the benefit derived from death? I'd suggest death is simply the terminal state that most systems find themselves in after a sufficiently long period of time has elapsed or they've experienced a set of inputs outside the range they were designed to handle and keep operating.
Post reply on HN