When to do asynchronous programming

Jose Maria Arranz has a post up about The “joy” of asynchronous programming. Jose’s key point is programmers are “reinventing the wheel”. By manually breaking tasks into a multi-threaded asynchronous/callback paradigm, programmers are reinventing the thread scheduling that the operation system is already doing. Jose’s point is that programmers should simply create as many threads as needed and let the operating system do its job. At the end of the day, everything funnels through the hardware and in practice the actual concurrency is dictated by the number of cores.

I agree with the thrust of Jose’s perspective. Programmers should operate at the highest level possible. Thread management is very low-level and best left to the operating system.

However, asynchronous programming is valuable when used correctly. The specific cases that come to mind are:

1. Batch Operations ( for example, sending email, or file transfers). Typically the user is not waiting on the completion of the operation. The user may no longer be logged in. So deferring actual execution of the operation is useful and does not create valueless complexity
2. Transferable Operations: operations that can be transferred to another backend server for completion. For example, map/reduce operations with the eventual result being reported to the user.
3. Optional Operations: Operations that give a better user experience, but under heavy load can be terminated and a default result returned. Netflix does this with user recommendations. When Netflix is under heavy load or if the user-specific recommendations are slow coming back, the operation is timed out and default recommendations re returned.
4. Speculative “Prefetch” Operations: Operations that are done in anticipation that the result of the operation may be needed soon.
5. Network Delayed Operations: If the operation requires connection to an external service for example, sending a calendar item to upcoming.org. Upcoming.org may be down or slow. Your site’s user experience should not be impacted because of delays outside of your control. There is nothing that can be done about upcoming.org’s behavior. So decoupling your users experience from external influences via an asynchronous programming model is useful.