I tend to just pay attention roughly to what I eat, and not eat foods that are generally bad for me/empty calories. However on some days I'll consume twice the amount of "recommended daily calories" for an adult my size. Yet I remain an 140lb adult male.
There are no hard/fast rules for calorie consumption, and I don't believe a calorie counter will help. In-fact, it is the precise quantification of food that leads to poor diets/eating habits.
EDIT: My fear is that focusing on only a small part of nutrition will further harm the nutrition of individuals. If we place too much attention on calories other things can be overlooked such as vital fats and acids or minerals. I'm trying to make the case that increased awareness and education of food and the human diet/nutrition is more important than counting calories. As someone points out below, there are many situations where calories are not labeled. In my case, I'm aware of what food is decent, and what food is junk and this allows me to forgo strict quantification of the food I consume. I know that food from most food chains is generally junk. This extends beyond the nutrition of the food as well. Food chains typically source their food from the cheapest distributors, and these are often the same distributors that end up distributing massive quantities of contaminated foods.
Here's another case. I could potentially consume 5 sodas in a single day and remain under my caloric restriction, but I would have consumed a ton of empty calories. A calorie monitor is not able to help in this case, but my own awareness of health and nutrition keeps me from doing this.
The issue of health is far larger simple quantification of a few metrics. It is within any individuals ability to determine for themselves general effects of the food they are consuming.
For some of the best arguments contrary to my own: http://www.foodpolitics.com/tag/calorie-labeling/