I used to be a true believer in the free market and I did want to abolish international borders to enable free trade of labor. What I didn't realize though is that nobody wants to require immigrants to pull their own weight and exclude them from social welfare if they're unemployed, etc. If you had a very free market country with no social services that would be overused by unrestricted immigration, then yes, an open boarder might be a good idea. Perhaps this is similar to internal borders in China, which are reasonably open but immigrants from other provinces aren't eligible for social welfare and effectively have to go back home if they lose their job.
Which is why the only rational position of a true believer in the free market is to abolish international borders.