Perhaps this 'energy' is actually what you feel in response to something you have perceived. So no actual energy is involved, it's just a colloquial term used to describe the intuitive feeling you can get due to your perception of a person, place or situation. I think this can sometimes be triggered due to something else we are processing internally at the time and we incorrectly attribute the feeling to the person we're talking to, place we are visiting, etc.
My thoughts, anyway. Good topic.
Another thought; Mythbusters did a test where the hypothesis was that a certain low range, inaudible (to humans) frequency of sound can cause a person to feel uneasy. The team went to some relatively identical abandoned cabins and set up a huge sound system behind one of the houses. They recruited some self-professed psychics and told them something bad had happened in one of the houses, and they would each, in turn, go through each house and see if they could detect which house it had occurred in. When each psychic got to the rigged building, they turned the equipment on, but I think only one person kind-of thought that cabin 'felt different'. So that myth was busted.