Summary: I examine the association between the observable physical world and the mathematical models of theoretical physics. These models will exhibit many entities that have no counterpart in the physical world, but which are still necessary for the mathematical description of physical systems. Moreover, when the model is applied to the analysis of a physical system, it will sometimes produce solutions that are unphysical ‒ i.e. describe a physical system that simply cannot exist in the real world. It is argued that this poses a problem for the epistemic position of realism in physics: a mathematical theory that professes to give a correct description of physical reality should not contain such unrealistic objects. Some concrete examples are examined, and their implications for physics are discussed from a relativist epistemic perspective. I argue that the appearance of the unphysical entities and solutions also has significance for the teaching of physics. It is recommended that the students be exposed from the start to a relativist ontology, as advocated by the theory of radical constructivism; here the unphysical objects do not pose an epistemic problem.