Programming

Experimenting with async endpoints in a Flask 2.0 project

PRG

Flask 2.0's async view support means you no longer need a separate framework, or an ASGI-only rewrite, to write async endpoints in Python. We migrated a few I/O-bound endpoints, mostly ones calling slow third-party APIs, to test the real-world benefit rather than take the changelog's word for it.

For endpoints dominated by waiting on external calls, response times under load improved noticeably. One endpoint that aggregates data from three separate vendor APIs went from issuing those calls sequentially to firing them concurrently with asyncio.gather, and the p95 latency dropped by more than half under our load-testing setup. That's the case async is built for: the CPU is idle almost the entire time a request is being handled, and letting the event loop pick up other work during that idle stretch is close to free performance.

For CPU-bound endpoints, async bought us nothing, which is exactly what the Flask docs warned would happen. A report-generation endpoint that does real computation showed no measurable improvement, and in one case got marginally slower once we accounted for the overhead of running it under an ASGI server instead of Flask's default WSGI setup. Async doesn't parallelize CPU work; it just avoids blocking on I/O, and applying it where there's no I/O to avoid blocking on is a wasted migration.

The migration itself was less disruptive than expected. Flask lets synchronous and async views coexist in the same application, so we didn't need a big-bang rewrite; we converted the slow, I/O-heavy routes first and left everything else alone. The main gotcha was making sure any database or ORM calls inside an async view were actually async-compatible, since mixing a blocking database driver into an async view defeats the purpose and can silently reintroduce the exact bottleneck you were trying to remove.

Getting the migration right required auditing every dependency the affected endpoints touched, not just the endpoint code itself. A couple of our internal HTTP client wrappers were built on the synchronous `requests` library, which blocks the event loop just as effectively inside an async view as a slow database call would; swapping those specific call sites to `httpx`'s async client was a prerequisite for the migration actually delivering the improvement the benchmark suggested it should, rather than quietly serializing everything behind the scenes despite the `async def` on the route.

We also had to rethink error handling slightly. A synchronous Flask view that raises an unhandled exception behaves predictably within Flask's existing error-handling machinery, but an exception raised inside an awaited coroutine needs the same handling to actually propagate correctly through the ASGI adapter Flask uses for async support. We added a small test suite specifically exercising failure paths in the newly async endpoints, since a silent failure to properly propagate an error would have been far worse than the performance problem we were trying to solve in the first place.

Load testing before and after the migration mattered more than we initially planned for. Our first attempt at benchmarking used a tool that itself became the bottleneck at high concurrency, producing numbers that looked like async had made things worse until we realized the load generator, not the server, was the limiting factor. Once we fixed the test harness, the improvement on I/O-bound endpoints was consistent and repeatable across multiple runs, which gave us enough confidence to roll the change out to production rather than treating the first benchmark as the final word.

Our takeaway is narrow but useful: profile before reaching for async, and only convert routes where the bottleneck is genuinely I/O wait. Treating async as a general performance switch would have wasted a lot of engineering time on endpoints where it can't help.

← Back to the journal

Have a project in mind?
Let’s talk.

Tell us where you are and where you want to go. We'll map the fastest route between the two.

Currently accepting new clients