It's a way of fulfilling the commandment in the Torah from the longer Shma:
Hear, O Israel: The Lord our God is one Lord:
And thou shalt love the Lord thy God with all thine heart, and with all thy soul, and with all thy might.
And these words, which I command thee this day, shall be in thine heart:
And thou shalt teach them diligently unto thy children, and shalt talk of them when thou sittest in thine house, and when thou walkest by the way, and when thou liest down, and when thou risest up.
And thou shalt bind them for a sign upon thine hand, and they shall be as frontlets between thine eyes.
And thou shalt write them upon the posts of thy house, and on thy gates.
Deuteronomy 6.
The mezuzah contains verses from Deut 6.