You should be comparing calories, not pounds. Also, it really depends on what vegetables you're comparing (there is a 50x range in the amount of calories provided per input for plants as a class). Beef is generally
less resource intensive than leafy green vegetables, but more resource intensive than most starchy vegetables. Chicken is less resource intensive than basically all plant food sources except for peanuts, potatoes, oils, and grains.
A brief glance at what most vegetarians and vegans are eating leads me to believe that their diets are probably not that much more environmentally friendly than your average meat eater. The real key is shifting toward inexpensive cheeses, milk, and oils for protein and fats and then consuming a lot more carbohydrates in the form of potatoes and grains.
And, really, the biggest impact environmental impact that most people can effect with their diet is not eating out as much.