No need for something as brittle as geographic coordinates or addresses. Time Zone names are the right abstraction. A given location very rarely [edit: adopts another] time zone (unless in case of annexations, handovers of territory, etc.), but time zone rules (zone offset from UTC, DST, etc.) change all the time. A consequence of this is that the time zone database of applications should be updated every few months!
"Complicated" then. One would have to consult a map to find out the time zone from geographical coordinates. The time zone is really everything needed to properly perform timezone-aware arithmetic.