Bread is fine much like most other foods. If you eat 2500 calories a day of broccoli you will gain weight. From broccoli.
American portion sizes being out of control is a problem. If you want to lose weight you can by all means order a sandwich. But if it's a honker then save half for lunch tomorrow, too. Two delicious meals for the price of one, and no pressured anxiety around ordering something you love to eat.