Ehh.... I worked for a long time as a mechanic on diesel engines. The engines are designed to conform with emissions requirements as they are written, not necessarily to be more efficient or cleaner. And of course some auto manufacturers have taken rather "creative" approaches to meeting these requirements.
Even with everything above board, the technology is not really there to make diesel a great solution for autos. First, the emissions equipment in modern diesel engines adds weight and reduces engine airflow, making them less efficient than they could be. In fact, today's small diesel engines are only 10-20% more efficient than gas equivalents, when in the past they have been over 50% more efficient.
There are three primary types of pollution that come from an internal combustion engine--carbon, which causes global warming; nitrous oxides, which cause smog, and particulate matter, which contributes to air pollution and also causes all sorts of nasty health problems. Diesel engines are better than gasoline on carbon emissions by virtue of being more efficient, although as I mentioned this gap is closing. Special catalytic exhaust equipment has been added to newer diesel engines to reduce nitrous oxides, but it's expensive and the environmental benefits are marginal. Particulate matter is addressed by burning the soot that comes out of the exhaust, again using expensive equipment that reduces efficiency, and AFAIK the science is still out on whether and by how much this actually reduces environmental/health impacts.
Some municipalities in the US have perhaps misguidedly banned diesel engines from their roads. The situation is a little different in Europe, where diesel engines are found in a much larger share of passenger vehicles. If Europeans drove as much as Americans did, their cities would all look like LA circa 1975. I'm not sure about car ownership trends in Europe, but I think it would make sense for any city that expects increases in car ownership to start regulating diesel more heavily.