Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A First Course in Differential Equations for Scientists and Engineers

people.uncw.edu

Post reply on HN
Print view View original

61–67 of 67 posts

Re: A First Course in Differential Equations for Scientists and Engineers

#61

Post by commandlinefan »

graycat wrote: post #27
jamescostian wrote: post #9

Earlier quoted context omitted.

> I really don't think that such topics can be self-studied Can you explain this? What makes something extremely difficult to be studied without a teacher? (I took calc in high school and never took diff eq, so my knowledge in this specific domain is basically zero)

With calculus, you are already well into ordinary differential equations -- partial differential equations are different, but with calculus you have a good start on the basics of those, too. A simple ordinary differential equation is below where of course just from calculus y'(t) = d/dt y(t) and the equation is y'(t) = k y(t) ( b - y(t)) So, for the context: t is time, say, in seconds. y is some real valued function…

> hanging over the whole subject is a suspicion that, really, as nice as the general theories are, mostly the applications are just a few, standard differential equations

I see something similar when it comes to machine learning. If you start to really dig into the underpinnings of the topic, you find fairly complex things like partial derivatives (for gradient descent optimization, for example), but you don't really have to understand it much to take it, apply it, and verify that the results make sense. On the other hand, I've been around long enough to have learned the hard way that applying something you don't really, fully, from-the-ground-up comprehend can bite you in surprising ways.

Re: A First Course in Differential Equations for Scientists and Engineers

#62

Post by takk309 »

mclehman wrote: post #25
takk309 wrote: post #20

Earlier quoted context omitted.

I could not imagine trying to take notes with a computer in a math heavy subject. I had to use paper for everything during my undergrad in Civil Engineering. However, if you already have a computer with you, record the lecture and review it later.

Just as a partial counterpoint to this and overall agreement with everyone else who responded, I've had success taking realtime notes on my laptop in a math class. It took a bit of practice and I'm sure I can't do it anymore, but it's not insurmountable by any means. I used latex and made liberal use of keyboard and software macros to do it, and one of the tricks was to realize that if I needed a quick-to-type way to…

Excellent counterpoint. Keep in mind that I was a Civil Engineering student, not computer science. My abilities to use Latex were little-to-none at the time. I didn't get any exposure to it until I was in grad school and we used it to format journal article submissions.

Re: A First Course in Differential Equations for Scientists and Engineers

#63

Post by graycat »

commandlinefan wrote: post #61
graycat wrote: post #27

Earlier quoted context omitted.

With calculus, you are already well into ordinary differential equations -- partial differential equations are different, but with calculus you have a good start on the basics of those, too. A simple ordinary differential equation is below where of course just from calculus y'(t) = d/dt y(t) and the equation is y'(t) = k y(t) ( b - y(t)) So, for the context: t is time, say, in seconds. y is some real valued function…

> hanging over the whole subject is a suspicion that, really, as nice as the general theories are, mostly the applications are just a few, standard differential equations I see something similar when it comes to machine learning. If you start to really dig into the underpinnings of the topic, you find fairly complex things like partial derivatives (for gradient descent optimization, for example), but you don't really…

I'd suggest learning about partial derivatives and in particular, the gradient. With an appropriate book, one evening should be enough. Intuitively the negative of the gradient is the direction to ski fastest downhill, and that's why it is heavily involved in optimization problems. Then there are close connections with convexity -- intuitively the inside of most kitchen cereal bowls is convex. The gradient is crucial in uses of Lagrange multipliers in the non-linear cases; in non-linear optimization with constraints, the gradient is central to the Kuhh-Tucker-Karush necessary conditions for optimality. If are in a deep, long, narrow river valley, want to get to the bottom downstream of the river, and ski downhill using the gradient, then will keep crossing the river over and over, traveling many feet across the river for each foot going downstream. So can approximate the valley with an ellipse and ski in much better direction along the long axis of the ellipse. People figured this out long ago -- it's called conjugate gradients. If the river wanders, then that's still more difficult. In the best fitting in ML, may be in such river valleys, and some notes on ML recognize this and warn about using just the gradient. A lot more with gradient is known and at times useful -- Newton iteration, quasi-Newton, etc.

Re: A First Course in Differential Equations for Scientists and Engineers

#64

Post by gmiller123456 »

nraynaud wrote: post #41
gmiller123456 wrote: post #6

While the linked course seems a lot more thorough, I took the Udacity "Differential Equations in Action" [1] course, which I found very well done. For the homework you write Python programs to compute things like gravitational slingshots, modeling epidemics, wildfires, and the n-body problem. [1] https://www.udacity.com/course/differential-equations-in-act...

Can we get to the text and videos without all the school-ish crud? (starting a course, timelines etc.)

Yes [1]. I haven't taken a Udacity course in quite a while, but at the time all of their videos were hosted on Youtube, so you should be able to find a lot of their courses just on Youtube. For the homework, I remember there was an include file that they didn't provide the source code for, but a Google search turned up a version someone wrote for using it locally. Usually it's pretty obvious when you've solved the problem and have the correct code, but to make absolutely sure, you have to upload your code to their server for verification. But you can take the whole class without ever visiting Udacity.

[1] https://www.youtube.com/watch?v=8cLXVG2Q6D4&list=PLAwxTw4SYa...

Re: A First Course in Differential Equations for Scientists and Engineers

#65

Post by nraynaud »

gmiller123456 wrote: post #64
nraynaud wrote: post #41

Earlier quoted context omitted.

Can we get to the text and videos without all the school-ish crud? (starting a course, timelines etc.)

Yes [1]. I haven't taken a Udacity course in quite a while, but at the time all of their videos were hosted on Youtube, so you should be able to find a lot of their courses just on Youtube. For the homework, I remember there was an include file that they didn't provide the source code for, but a Google search turned up a version someone wrote for using it locally. Usually it's pretty obvious when you've solved the pr…

thanks!

Re: A First Course in Differential Equations for Scientists and Engineers

#66

Post by srean »

graycat wrote: post #63
commandlinefan wrote: post #61

Earlier quoted context omitted.

> hanging over the whole subject is a suspicion that, really, as nice as the general theories are, mostly the applications are just a few, standard differential equations I see something similar when it comes to machine learning. If you start to really dig into the underpinnings of the topic, you find fairly complex things like partial derivatives (for gradient descent optimization, for example), but you don't really…

I'd suggest learning about partial derivatives and in particular, the gradient. With an appropriate book, one evening should be enough. Intuitively the negative of the gradient is the direction to ski fastest downhill, and that's why it is heavily involved in optimization problems. Then there are close connections with convexity -- intuitively the inside of most kitchen cereal bowls is convex. The gradient is crucial…

Vigorously seconded. There is a lot of applications of convex analysis, convex duality, KKT conditions and game theory in ML if one looks at it right. In fact the thats at the very foundation of techniques such as support vector machines (large margin separators in a Hilbert space), regret minimization algorithms, etc etc.

Of course one can choose to ignore all that and only focus on stochastic gradient descent. That will carry one for some non-trivial distance.

Re: A First Course in Differential Equations for Scientists and Engineers

#67

Post by graycat »

commandlinefan wrote: post #61
graycat wrote: post #27

Earlier quoted context omitted.

With calculus, you are already well into ordinary differential equations -- partial differential equations are different, but with calculus you have a good start on the basics of those, too. A simple ordinary differential equation is below where of course just from calculus y'(t) = d/dt y(t) and the equation is y'(t) = k y(t) ( b - y(t)) So, for the context: t is time, say, in seconds. y is some real valued function…

> hanging over the whole subject is a suspicion that, really, as nice as the general theories are, mostly the applications are just a few, standard differential equations I see something similar when it comes to machine learning. If you start to really dig into the underpinnings of the topic, you find fairly complex things like partial derivatives (for gradient descent optimization, for example), but you don't really…

> If you start to really dig into the underpinnings of the topic,

Let's cover the most important part of partial derivatives, the geometric intuition.

Imagine the Smoky Mountains of east Tennessee, that is, smooth, rolling hills. Now to represent this landscape in math, let R be the set of real numbers, R^2 pairs of real numbers, that is, the coordinates of points in the plane with orthogonal axes X and Y, and let f: R^2 --> R, that is, f is a function of two variables, say, x and y, that is, the pair (x,y) in R^2, and the value f(x,y) is the height of the mountains above point (x,y), that is, a plane under the mountains.

Then at a point (x, y) the partial derivative of f(x,y) with respect to x is just the slope as in ordinary derivative of the mountain at point (x,y) in the direction of changing x. So, if the X axis runs east and west, the partial derivative of f(x,y) with respect to x is the slope of the mountain at (x,y) in the east-west direction. So the partial derivative is just like the derivative of a function of one variable, that is, a slope, except is for just one variable, say, x, with the other variable(s) y held constant.

So, if

f(x,y) = 3xy + 2x - y

then the partial derivative of f(x,y) with respect to x is just

D_x f(x,y) = 3y + 2

and the partial derivative of f(x,y) with respect to y is

D_y f(x,y) = 3x - 1

These partial derivatives are important in vector analysis and, thus, Maxwell's equations, electro-magnetism, fluid flow, optimization, etc.

Post reply on HN