Uh, yeah. That's the entire point. That's like saying "The only reason you don't like murder is because people get killed.".The only reason you're crying is because he owns the place.
We don't like the conflict of interest because it's a conflict of interest. If Trump didn't own the place, then it wouldn't be a conflict of interest.