> * perhaps LLMs can build such a theory from existing codebases, or will be able to in the future
If the source code is a lossy representation of the theory, doesn't that answer this question with a conclusive "No"? (If there is anything -- LLM or not -- that can do this, then the source code is not lossy after all.)
I guess that's right, but by "such a theory" I didn't mean the original theory - I meant maybe they can figure a different one out that works well enough, at least for the LLM. This is all handwavey, of course.
If the source code is a lossy representation of the theory, doesn't that answer this question with a conclusive "No"? (If there is anything -- LLM or not -- that can do this, then the source code is not lossy after all.)
EDIT: Clarify last sentence.