I think you have a good point, space dust would not be good let alone smaller pieces.
I thought I figured out the solution about 20 years ago for that. Having the craft's extrerior with light as energy passing through sort of fiber optics on the skin of the craft. One could put out less energy in front then the rear of the craft making a sort of shield of energy so to speak.
For larger pieces one could have a tracking device that would put a small sharp focused beam in any direction to steer unseen objects away from the path of travel.
Oh well just fun thinking about stuff like this letting imagination fly. If the religious can do it on a daily basis scientist can do it as well