Change your diet, but only make changes that you can see yourself following for the next 10 years. The main exception is verified health concerns (ex: Control your blood sugar if the doctor has told you too).
What this really means in practice: You won't be able to cut all sweets and treats, but surely you can commit to having them less often. You can eat less meat and more vegetables. Eat more legumes and more whole grains when you can stomach them. These changes eventually become normal, and really there is no failing. At the end of the week, if you are able to do this 4-5 days, you are better off than if you didn't. Always focus on a better overall diet. And in general, doing this means fewer calories consumed. Reasoning for this: If you eat more vegetables (for example), those vegetables generally replaces higher calorie foods. Since nothing is realistically off limits, the times when you really miss having (favorite food) are minimized.
You'll also probably want to eat less. For me, this means eating more at a time of day when I'm the most hungry: Evenings. I eat little during the day, and more at my evening meal. As a bonus, I feel completely satisfied (full!) after my evening meal, and that is a wonderful feeling.
If you can, work in more accidental activity. Walk to the store if one is close enough, for example. Walk the dog more, play with children in active ways, and so on. These don't seem like exercise, per se. Of course, if you like an exercise or activity, by all means, do that more.
Edit: I almost forgot to add to get rid of sugary drinks in general. This includes diet drinks - the taste sugary, and this is what you should get away from. I personally tend to drink only water or black coffee on a normal day. When I drink soda and other things, I tend to drink the fully sugared one. Also, if you drink alcohol, you probably have calories there.