When I first started practicing Hinduism, I read every book I could get a hold of on the religion, from Hindus and non-Hindus alike.
Of what I read, I'd say that by far the most important modern works to read for a newcomer would be the works by Swami Vivekananda, especially his essay "Paper on Hinduism", which is part of his "Addresses at the Parliament of Religions", and his book, "Karma Yoga."
Since he lived 100 years ago, his works are freely available online (or you could buy the whole, 10 volume set for 10 dollars if you, like me, prefer holding a physical book in your hands. ^_^):
Complete Works - Index - Volumes
Since you seem drawn to Krishna specifically, I recommend Vaishnava works, specifically from the Gaudiya sect. I, myself, wasn't a Vaishnava (I was drawn more to Kali and Shiva), so I don't have an exhaustive knowledge of appropriate literature, but anything by Sri Chaitanya is an absolute
must. Plus, read the two main Vaishnava Puranas: the
Bhagavata Purana (I recommend against Sri Prabhupada's translation/commentary for a beginner; I preferred Swami Tapasyananda's, personally), and the
Vishnu Purana, which is available at Internet Sacred Text Archive for free. If the unabridged works are too intimidating (any given Purana is about the length of the Bible), there are plenty of abridged versions.
Finally, familiarize yourself with the stories of the Ramayana and the Mahabharata.