Hmm, good point. How would you separate the two? I typically equate them to being similar.
I tend to see "fit" as being related to one's ability to do physical exercise activities, like weight lifting or running a mile.
Being "fit" is a component of "physical health," but "physical health" includes the whole spectrum of the body. It includes things like whether or not various vitals are within a designated healthy range for your sex and age bracket. It includes things like whether you've got a chronic or acute medical condition, whether it's something like cancer, a sprained ankle, or the flu.