No, they impact a certain number of people as a statistical measurement and distribution. A certain number of people will have certain advantages in various situations because they are white, and a certain number of people will not. A certain number of people will have disadvantages in various situations because they are black, and a certain number will not.
These numbers will, of course, vary widely according to a whole range of other variables. However, the interesting, and useful component of this is to be able to see the statistical variance of these numbers, and to try to isolate all the variables and influences to be able to formulate productive and effective solutions, in regards to bias and its effects.
But this has absolutely nothing to do with the real-world application of "privilege" or "disadvantage" both being generalizations. The social drive to paint all white people as having some all-pervasive and ever-present "privilege" is not only inaccurate, but not very useful. Nor, is the opposite drive to paint all minorities as have some all-pervasive and ever-present disadvantage.
Of course, I don't really expect most people coming from a purely social protest ideology to necessarily put much emphasis on either accuracy, nor usefulness. I extend this to all people who are primarily ideologists, particularly in the reactive sense.