Every brain creates a model of reality that you then experience. The general hardware is always the same, neurons, glial cells, etc. But they are not identical in any two people, and so they will generate different models.
We have evolved a "toolkit" for processing wavelengths of light into colour, which can then be subjectively experienced. But how, exactly, you experience it really depends on the unique structure and patterns of your brain cells. There is a "program" your brain runs to produce this subjective experience of colour, and everyone has roughly the same program. But there are myriad small differences (and sometimes large ones) in how the program runs in each brain.
These differences ensure that there will be, at the least, subtle variations in the qualia of the colour in each person's mind.
What we are experiencing is not reality, it is a model of reality, according to neuroscience. As a model it doesn't represent reality exactly, but well enough for animals like us to survive and function with.
But, because it is a model, there's nothing to ensure that we see colours the same, qualitatively. Such a thing would have no reason to evolve. Our brains have a fixed number of possible colours that can be incorporated in the model (1 million colours in males and up to 100 million colours in females). Which of these colours get incorporated into your model of reality and subjectively experienced is not set in stone. Everyone's experience of their own model's qualia will be slightly different.
It's quite possible that variations in subjective experience of qualia are what produce our unique emotional and physiological responses to colour. These responses tend to be bound (for past evolutionary reasons) to be at least sort of similar, but, again, there will always be, at the very least, subtle differences.