I don't use Swift not because I think it's not useful outside Apple, but because I believe its developer experience is poor. Some stories that formed my opinions:
what advantages does swift offer over go/rust/js/java for server side programming? I always presumed the advantages of swift were native code compilation + top tier integration w/the apple ecosystem.
I don't think people care about its relation to Apple, they care about the language ecosystem, roadmap and evolution that were shaped by Apple's needs for iOS and macOS before there was a real attempt at making Swift more general purpose and multi-platform. And now that it's somewhat there, there are better options in almost every dimension.
I have a few personal and professional Swift on server projects, in the wild and in the works. Code reuse is a big win – we can ~easily expose functionality of our client apps to other systems. Familiarity is another – there's an ocean of iOS (and, to a lesser extent, macOS) developers out there who are familiar with Swift. With a little bit of coaching, they can pretty quickly get up to speed with how services work.
It reminds me a lot of what it was like to ship Node.js software 15 years ago – the ecosystem is still young, but there are enough resources out there to solve 95% of the problems you'll face.
In my experience, writing Swift for the backend feels a lot like writing TypeScript, but nicer — though that’s just a personal preference. You get the performance of a compiled language like Rust (though that’s rarely a bottleneck for backend applications), but Swift is significantly easier than Rust and has much faster compile times.
An example: https://vapor.codes/
The problem is that people only think it’s generally useful in the Apple ecosystem.