Why is reality mysterious? Why should difficult questions persist for so long despite the successes of physical science?
An answer to these meta-questions may lie in the concept of a phase transition. As discussed in prior posts (like this recent one), a school of quantum gravity research has arisen which explores the idea that the visible cosmos (of matter bound in space-time geometry) arises at lower energies from a more fundamental quantum world. This more fundamental level is usually characterized by a network of quantum systems, subject to a directional causal arrow, but otherwise connected in a highly non-local fashion (little or no recognizable spatial geometry).
This model is inspired by the myriad examples of phase transitions observed in nature, and particularly those in the field of condensed matter physics (which utilizes the toolkit of quantum field theory to describe the phenomena). Superconductors, superfluids, etc. display remarkable emergent features which arise under certain pressure/temperature conditions.
Picture our familiar physical cosmos as a portion of reality which condensed into a “classical” phase, but retained subtleties in its nature which reflected its pre-transition roots. If this analogy works, then it would explain our situation: while classical explanations usually work well, some phenomena defy such analysis because their foundations go deeper. This could be the case, for instance, for the arrow of time and for conscious experience itself.