If one were unconstrained by development effort and sales, concerned only with deep transferable understanding, I wonder what one might explore with representations that were less cartoonish, more physically realistic?

Directly rendered electron density? With "bonds" implicit. Plucking, distorting, ripping, and assembling molecules grabbed by their nuclei. Or poked with others. With realistic and explicit time/space scale and dynamics. Captures of ab initio simulations, with their rich complexity and ambiguity.

Nuclear density, with alpha clusters, liquid drops, interesting shapes, and again, a hands on realistic dis/assembly and messing around.

It's pragmatically hard now. But arguably possible. So now, or years from now, there exists the question of how might it be used, how much might it help. How hard should we be trying to make it happen some years sooner rather than later. But we don't seem institutionally set up to explore such questions.

> google daydream grade VR

Yeah. I'm so very looking forward to the next year or three of hmds, especially AR. Wait, "in realtime"? I'm mostly bottlenecked on angular resolution. What challenges are you facing?