Better many copies than none. References usually mean written text and maybe some figures, cost of storage is going down, we can afford the duplication.
> everyone who references the same thing is then keeping a copy of it.
... and it would serve as a form of redundancy, imitating the fungible nature of physical media: In order to cite the latest, copied, manuscript (for example) you needed to own a physical copy. They existence of these has enabled survival of works that would otherwise have been lost, or even reconstruction through ecdotics.-
Books deserve a github repo with PDF web archives of referenced links, the same way that Wikipedia mirrors the content of cited links.