I think that's right. At some level, any anticipation of a future state has to be measurable in some kind of confidence level.
I suppose where I get lost is that, at least subjectively, I end up treating different anticipated return distributions differently. I want a mixture of risks, both in terms of their covariance and the absolute properties.
When I think of "speculation", I am intentionally shaping only a small portion of my personal portfolio toward high risk, high reward activities. And I only really feel comfortable doing that because a larger fraction of my personal risk is in safer vehicles.
Atop that, I also think a lot about liquidity time bounds. I want access to a reasonable amount of highly-liquid, low-risk investments. I need that flexibility to be safe in the event that I need to buy something.
To my eye at least, I qualitatively differentiate between speculative investments and these liquid/low risk ones. If I felt I only had one kind of risk, I would seek out the other in some proportion.
You don't know what will happen in any of the cases. You just choose to believe one more over the other based on what has happened in the past.