I don't know if I'm being clear. Say you and your family were imprisoned. You would never demand to be released by saying "let's go!". Your bemused family might well ask "Where, to the other corner of the cell?"
English contractions are weird in general in that it doesn't always "work" to contract two words. Tom Scott does a good video about this: https://www.youtube.com/watch?v=CkZyZFa5qO0
(Example, "Is this a good idea? Yes, it's!" sounds wrong. But "it's" still means "it is". It would just sound weird to use a contraction in that context.)
Somebody else brought up the example of “let’s go!” versus “let us go” - not the same thing by a long shot.
“Let’s” in English has a distinct meaning from “let us”, and that is to politely and casually (but firmly) suggest a course of action.
I remember touring a Polish salt mine a couple of years ago. The guide was very good, but her English had a few quirks, among them that she seemed to like the phrase “let’s let me to show you …”. It’s wrong, but you can immediately understand that she meant “please let me show you”.