Earlier quoted context omitted.
Awesome, thank you! Interesting. The first thing that came to my mind regarding the traffic light example is any problem that reduces to a SAT solver, I assume they are some that are clearly un-smoothable in polynomial time otherwise this will have interesting consequences...
I agree with that intuition. In our experience, it's easiest to see gains over other optimization techniques when the program is "branch-wise smooth and non-constant". Then, we get the full benefits of exact autodiff gradients "per branch", and our smoothing approach handles the branches. For SAT solving and other purely combinatorial problems, sufficiently accurate smoothing may indeed be too expensive. Also, in suc…
Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
11–20 of 67 posts
Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#12Talk about the differences between this and ceres http://ceres-solver.org/ .
While I'm not super familiar with the typical use cases for Ceres, the gradient estimator from DiscoGrad could possibly be integrated to better handle branchy problems.
Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#13When all you have is a hammer, ... you split up your house into wood and nails, and reduce it to a previously solved problem! In all seriousness, this is super interesting. I really like the idea of implementing gradient descent solving branch by branch, and turning it into an optimization-level option for codebases. I feel like this would normally be something commercialized by Intel's compiler group; it's hard for…
We were thinking of some disco ball-based logo (among some other designs). With this encouragement, there'll probably be an update in the next few days :)
Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#14Discograd: a little-known top secret Soviet project where Brezhnev wanted to counter American influence in global popular music by creating an entire military town dedicated to evolving the four-on-the-floor beat towards Marxist-Leninist perfection through cybernetic principles of rhythm composition. (After 1991 Discograd was demilitarized and renamed Grungetown to attract foreign investments.)
In this reality, Discograd hosted the first Soviet Rock Festival, which was attended by thousands of enthusiastic fans from all over the USSR. The festival featured performances by bands that were formed and nurtured in Discograd, showcasing a new genre: Proletrock – a unique fusion of disco, rock, jazz and Soviet folk music, with lyrics promoting socialist values and workers’ rights.
Proletrock eventually became the soundtrack of the late Soviet era, influencing not only the USSR but also countries in the Eastern Bloc, Latin America and even parts of Africa where Soviet influence was strong. The genre helped to spread communist ideology through catchy beats and thought-provoking lyrics, making Discograd an integral part of music history.
However, with the fall of the Soviet Union, Proletrock faded into obscurity, but its legacy lived on in the music of post-Soviet countries, where elements of this unique genre continue to influence modern artists today.
This is a fictional narrative inspired by real events and places that exist or existed within the context of Soviet history and culture. It serves as a creative exploration of what could have been if the USSR had pursued such an ambitious project with the same fervor it dedicated to its space program.
(WizardLM-2-7B)
Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#15Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#16Discograd: a little-known top secret Soviet project where Brezhnev wanted to counter American influence in global popular music by creating an entire military town dedicated to evolving the four-on-the-floor beat towards Marxist-Leninist perfection through cybernetic principles of rhythm composition. (After 1991 Discograd was demilitarized and renamed Grungetown to attract foreign investments.)
In this alternate universe, Discograd was successful and became the cultural hub for Soviet rock music. It influenced bands like Kino, Aquarium and DDT who incorporated Marxist-Leninist themes into their music. Disco beats were indeed evolved but in a way that resonated with the ideology of the time. The four-on-the-floor beat became more intricate and complex, incorporating elements of jazz and folk music from vario…
Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#17How does this compare to Enzyme ( https://enzyme.mit.edu/ )?
Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#18Can you talk a little bit about the challenges of bringing something like what you’ve implemented to existing autograd engines/frameworks (like the ones previously mentioned)? Are you at all interested in exploring that as a mechanism for increasing access to your methodology? What are your thoughts on those autodiff engines?
Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#19I remember being taught how to write Prolog in University, and then being shown how close the relationship was between building something that parses a grammar and building something that generates valid examples of that grammar. When I saw compiler/language level support for differentiation, I the spark went off in my brain the same way: "If you can build a program which follows a set of rules, and the rules for that language can be differentiated, could you not code a simulation in that differentiable language and then identify the optimal policy using it's gradients?"
Best of luck on your work!
Re: Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad
#20Awesome! Several of my colleagues are working on differentiable physics simulations (mostly FEM type stuff for structural design optimization) so I’m excited to share this with them! They mostly work in Julia. My own experiments with auto-diff’d physics sims have been in Python (specifically, Taichi for the JIT/GPU acceleration or occasionally PyTorch/Jax). Can you talk a little bit about the challenges of bringing s…