Right, and it's their right to choose to do so. Just as it's your right to cook strong smelling foods, or run an engine which outputs toxic gases.
The idea that second-hand smoke dispersed over the area between two separate houses is dangerous is rather ridiculous, as even the science behind the effects of more direct second-hand smoke are flimsy at best.
So, essentially it comes down to discomfort caused by a smell you don't like, such as a neighbor cooking foods whose odor you find objectionable.
In regards to the neighbor that smoked the cigar, you have a point, which is why I haven't, nor will I, say anything. I'm tolerant and fair.
The construction worker, however, was too close to my home, and despite that, I still said nothing, closed my window on that side of the house, and carried on with my day.