Environment being another source of information was mentioned (in my 2nd paragraph), but interactions that are determined by their starting points don't add information - even if the result looks more complex. This idea (and the comment about the mandlebrot set) is similar to "
pi holding all the information in the world" (assuming it's normal, meaning not repeating/regular)
http://news.ycombinator.com/item?id=1567225Whereas the training data for a neural net is extra information - but in utero, what is the training data that is not a predictable consequence of the genome? (ex utero there might be an argument, since humans not exposed to language don't develop it; although a group of isolated humans have developed language spontaneously - complex grammatical structures, the whole bit - which makes sense, given the variety of human languages. This supports the idea that language is genetic, or as Pinker provocatively describes it, the language instinct).
EDIT here's a thought experiment to illustrate why predictable interactions don't add information: taking the figure of six billion bits for the whole human genome, this means it can specify 2^6,000,000,000 different genomes (a lot). You can imagine changing one single bit, and all those complex interactions leading to a slightly different human phenome. Most of the possible phenomes wouldn't be a living human, or even anything recognizably human (or living). But the crucial point is that you simply can't specify any other phenomes (apart from those 2^6,000,000,000). You've changed all the bits - what else is there left to change within the genome?