Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Ok I think I get it, but if your backend is say, making a single API call and sending that result back to the user, then there's no advantage to going async since the user has to wait for that network call anyways, right?

Would you use this async then in instances where some of the API calls don't matter to your user?



I don’t know about Starlette/FastAPI, but generally a true async server could still see a benefit, because while that downstream call is blocked waiting on a response, the OS thread can switch to a different request. This is in contrast to a more traditional one-OS-thread-per-request model, where each thread spends the vast majority of its time waiting on I/O. If your work is I/O bound you can use expensive OS threads much more efficiently. Depends on your workload characteristics and requirements.


Python wsgi servers are generally based on processes and not threads since threads would trigger GIL considerations. While your process is making the single API call, the process is not serving anyone else, ergo you can only have so many simultaneous requests in flight as you have workers. Threads have no benefit over asyncio in python due to the GIL and have costs over ascyncio due to setting up the thread context with the OS.

With asyncio based implementations, while the coroutine is waiting for the response on the API call, the process's event loop can check on the coroutine which is handling the network socket and start processing another request. Ideally it should have no impact on the amount of time the original request takes to process but you can see how it might since when the API call resolves, the process may be working on the other request instead of continuing with the original coroutine.

Asynchronous code in general can allow you to extend your bandwidth past the number of concurrent executions your program can maintain (whether they be processes in python or threads in other languages) by doing additional work while it is waiting for something to complete. Ideally it should not cost much in terms of latency.


Without going asyncio you can always use greenlets and get the same benefits. Using uwsgi with gevent makes it easy.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: