One other comment: mathematics is full of what could be designated as 'useful fictions'.
For example, ALL of mathematics ultimately is based on set theory. But as it is developed, we have very different definitions of, say, the natural numbers (0,1,2,3,..), the integers (..,-3,-2,-1,0,1,2,3,...), the rational numbers (fractions), the real numbers (intuitively, all possible decimals), and the complex numbers (including the square root of -1).
The actual definitions given do NOT have the natural numbers as a subset of the integers. But, there is a 'useful fiction' that allows us to *think* about the natural numbers that way. Since all we are interested in is the relationship, such useful fictions can be quite helpful as long as they preserve all important structures (and what is important depends on the context).
So, there are several very different notions of 'infinity' in mathematics. There are limits, where no 'actual' infinity is ever required. There are cardinals, which essentially look at the 'size' of a set, where there are infinitely many different sizes of infinity. There are 'added points' where some structure is supplemented by an additional point (or several points) called a point 'at infinity'. There are ordinals (representing the order structure as opposed to just the size).
If wanted, I could come up with quite a few other examples, but these are the primary contexts for the use of the symbol of 'infinity'. Some are 'useful fictions', others are actual mathematical objects. And, there are contexts where those not aware of how math is done *think* it might be done and are just wrong.