Live data from Hacker News

The Mathematical Hacker

evanmiller.org

21–30 of 135 posts

Re: The Mathematical Hacker

#21
Math seems extremely important to me as a programmer, even for CRUD apps.

Let's say you graduated HS and slept through your algebra classes. Now a few years later you're a programmer with pretty much no math skills other than what you've learned in elementary school.

Math seems to teach you one of the most fundamental and useful things about programming. The ability to reduce a problem from something that is complex into something that is not complex.

I've only gotten a taste of some basic algebra after taking some online CS courses and really, coming into it with a background of "embarrassingly poor math skills" I can really say that it has changed my life for the better.

I'm still clueless when it comes to some algebra but I find myself looking at problems and being able to solve them much easier now and this is only after a few weeks of programming related courses that happen to use algebra on some occasions.

The math isn't what made it easier. It's applying the same things to solve math problems to programming.

Re: The Mathematical Hacker

#22
post #19

"Despite the aesthetic virtues ascribed to functional programming, I find the preceding solutions to be more beautiful than their recursive counterparts. They run in constant (rather than linear) time, and they are easily adapted to work with non-integer inputs." Isn't this wrong? I don't think pow is computed in constant time.

I think that it can be implemented in constant time for floating point numbers. For integers I know that the exponentiation by squaring takes O(log(n))

Re: The Mathematical Hacker

#23
post #19

"Despite the aesthetic virtues ascribed to functional programming, I find the preceding solutions to be more beautiful than their recursive counterparts. They run in constant (rather than linear) time, and they are easily adapted to work with non-integer inputs." Isn't this wrong? I don't think pow is computed in constant time.

I think many floating point functions (e.g. sin, cosine) are implemented using Pade rational approximations - basically, the ratio of two polynomials. (http://www.dattalo.com/technical/theory/sinewave.html)

This usually gives enough accuracy for the purposes of floating point.

However, I'm not sure if "pow" can be usefully implemented this way. I am guessing no, because pow grows faster than any polynomial eventually...

edit: Hmm, pow at least looks linear here: http://www.netlib.org/fdlibm/e_pow.c

Re: The Mathematical Hacker

#24
Picking on Fibonacci of all things? The goal of fibonacci and factorial examples are to teach recursion. Both fibonacci and factorial are good starting points for a beginner. It can be followed by discussions of dynamic programming where the student can be introduced to recurrence relations and solving them top-down and bottom-up.

EDIT: Adding some background on dynamic programming

For dynamic programming, the problem should be breakable in terms of overlapping smaller problems, and the base case should be recognized. If the problems don't overlap, they fall within broader divide and conquer category(mergesort, quicksort etc are famous examples).

fib(n) is defined as fib(n-1) + fib(n-2) (overlapping smaller subproblems) and fib(0) = 1 and fib(1) = 1 (base cases)

    fib(n) = fib(n-1) + fib(n-2)
    fib(0) = 0
    fib(1) = 1
A relation defined as above(recursively) is known as recurrence relation. Discrete math courses deal with finding closed form expression - a non-recursive function of n. But in programming, we are fine with solving the recurrence relation without finding a closed form expression.

Recurrence relations form the basis of dynamic programming and they can be solved either top down or bottom up.

The top down approach is the traditional recursive solution.

    def fib(n):
      if n == 0 or n == 1: return n
      return fib(n-1) + fib(n-2)
And then the student is to realize fib(n-1) is recalculating fib(n-2) and memoization is in order.

    def fib(n):
      cache = {0: 0, 1: 1}
      def _fib(n):
        if cache.has_key(n): return cache[n]
        cache[n] = _fib(n-1) + _fib(n-2)
        return cache[n]
      return _fib(n)
Then the student should realize modifying every function isn't apt, and should implement a general memoize decorator.

EDIT: Adding table based bottom up fibonacci.

Now once the student understands top down dynamic programming, as in he can find the recurrence relations and base cases, it's time for bottom up. As the name suggests, bottom up starts from the bottom and calculates n compared to top down which starts from n and boils down to base cases.

    def fib(n):
      vals = {0: 0, 1: 1}
      for i in range(2, n+1):
        vals[i] = vals[i-1] + vals[i-2]
      return vals[n]
Student should recognize how top down and bottom up are calculating the same recurrence relation, but in a different order. The table vals here is the same as cache above in top down.

Top down is recursive and might trigger the recursion limit. Bottom up doesn't have the recursion problem. Sometimes in case of bottom up, table can be eliminated depending on the overlap. But the important thing is, once the recurrence and base cases are known, it can be implemented quite easily.

In fibonacci's case, nth number depends only on n-1 and n-2 and maintaining the whole table is wasteful. The bottom up approach will be better.

    def fib(n):
      f0, f1 = 0, 1
      for i in range(n-1):
        f2 = f0 + f1
        f0, f1 = f1, f2
     return f2
Fibonacci just happens to be one of the problems used to demonstrate recursion and dynamic programming. It's small enough for a beginner to comprehend, and big enough to explain recursion and dynamic programming.

The article picks one recurrence which has a closed form expression. The dynamic programming problems which I have encountered aren't that easily reduced to closed form expressions.

Also, I don't know about Graham or Raymonds, but Yegge advocates maths for programmers.

http://steve-yegge.blogspot.in/2006/03/math-for-programmers....

Re: The Mathematical Hacker

#25
post #8

I don't really see the point of this article. You don't need Fortran or C to implement the calculations of the Fibonacci as described in the article. On top of that the article is missing the point that some elements functional programming (map, lambda) are actually making a numerical implementation neater. It is not an accident that the original authors of R were themselves lispers and admitted having been inspired…

The article didn't claim that FORTRAN or C are required for these calculations. Rather, It lamented that the prevailing attitude of Lisp practitioners was to eschew the knowledge behind those calculations.

Re: The Mathematical Hacker

#26
post #6

I find it disturbing that this article has been written in 2012. While I was reading it I really thought that it was at least 10 years old. Computer science researchers are doing actual mathematics, and they are clearly more in what the article calls "Lisp school" than the "Fortran school". Research in functional programming is mostly mathematics. Lambda calculi (which was originally not developed to be a programming…

Math is a very broad field. The point of this article is that applied math is generally not done in functional programming, and is not typically part of computer science research (more often happens as dedicated applied math research).

Types, logic, and category theory are the sort of things the author thinks that functional programming people concern themselves with, and he is raising the point that these mathematical concepts only help people create better languages and write safer code, not do "useful" things like weather/weapon simulations, solve optimization problems, or image/video processing.

Re: The Mathematical Hacker

#27
post #14
post #6

I find it disturbing that this article has been written in 2012. While I was reading it I really thought that it was at least 10 years old. Computer science researchers are doing actual mathematics, and they are clearly more in what the article calls "Lisp school" than the "Fortran school". Research in functional programming is mostly mathematics. Lambda calculi (which was originally not developed to be a programming…

At university in 2005 I studied Mathematics and Computer Science in my first year, hoping for a path into theoretical computer science (e.g. complexity theory). I couldn't find one; mathematics had its discrete side, and made use of programs in many areas, but was uninterested in the theory of computation; computer science was mostly java training (I chose mathematics). So I'm pleased if such research is going on, be…

> I couldn't find one; mathematics had its discrete side, and made use of programs in many areas, but was uninterested in the theory of computation; computer science was mostly java training (I chose mathematics).

I don't understand why schools do that. There is so much to know and learn in computer science course, yet somehow learning j2ee takes precedence above everything else.

Re: The Mathematical Hacker

#28
post #20
post #6

I find it disturbing that this article has been written in 2012. While I was reading it I really thought that it was at least 10 years old. Computer science researchers are doing actual mathematics, and they are clearly more in what the article calls "Lisp school" than the "Fortran school". Research in functional programming is mostly mathematics. Lambda calculi (which was originally not developed to be a programming…

Ironically, though, FP is perhaps the worst programming paradigm for actually performing numerical computations.

Mathematica borrows heavily from FP and is well known for its computational performance. And yes, a lot of its guts are imperative, but most of the internal libraries that encapsulate the actual mathematical concepts are written in Mathematica itself.

Re: The Mathematical Hacker

#29
The problem with the article is that it dreams up a false conflict and then buries its thesis, a very valid and solid one, by trying to force a contentious narrative onto that just isn't there. Very few people these days are ignorant of the points he makes and neither Yegge nor Graham are posts of some imagined counter applied math camp. I would tend to think they are pro - Yegge's interested in bio, Graham did spam stuff.

Here is my attempt at a summary that captures all the detail:

Programmer math is not only restricted to type theory and formal logic. Applied Math is also a very important aspect. Eric S Raymond is wrong for trying to make it look like the 'Except for X' is negligible. Plus, machine learning, bio, geo and so on are increasing in importance and growing rapidly as fields. Even in the past, applied Math Programs had a massive impact on the economy by enabling engineers with specialized software. Learn Math. It is important for you as a programmer going forward.

---------

I am not sure why he restricts functional programming to recursion with lisp. There is no need to denigrate functional programming, it fits extremely well with these applied math problems and helps a lot with reducing complexity of implementation, in my opinion. They also tend to be at least as fast as Java with OCaml posting incredible single core speeds.

Another point is math algorithms is distinct from mathematical programming. Math helps one calculate and choose faster algorithms or find better bounds. But except for some portions of the haskell compiler there are few uses of direct calculation to derive programs mathematically. In engineering, math allows you to precisely calculate behaviour and properties but in programming you have to debug. The key reason for this divide is that other engineers have lots of components with well defined properties to work with.

This is another reason why functional programming is more mathematical. Although the idea of composition is not inherent to FP, in FP it is the default paradigm. The ideas of combinators with well defined properties, and theorems proven on them, that only go together in a certain way and that you can sit down and have a pretty good idea of how your program would behave theoretically, this idea is what makes FP so close to doing applied math. And Haskellers really shine at that. Haskellers like to talk about monoids and categories but really most of haskell programming is closer to what a reguler engineer does with the category theoriests serving the same function as physicists.

Re: The Mathematical Hacker

#30
post #27
post #14

Earlier quoted context omitted.

At university in 2005 I studied Mathematics and Computer Science in my first year, hoping for a path into theoretical computer science (e.g. complexity theory). I couldn't find one; mathematics had its discrete side, and made use of programs in many areas, but was uninterested in the theory of computation; computer science was mostly java training (I chose mathematics). So I'm pleased if such research is going on, be…

> I couldn't find one; mathematics had its discrete side, and made use of programs in many areas, but was uninterested in the theory of computation; computer science was mostly java training (I chose mathematics). I don't understand why schools do that. There is so much to know and learn in computer science course, yet somehow learning j2ee takes precedence above everything else.

I suspect that has to do with businesses telling the university what they need out of a CS graduate. Certainly happened when I was going to school. Interestingly, the guys doing the fascinating math and research were in Electrical Engineering, CS was for programmers.
Post reply on HN