Sample code provided in this chapter is available on Github (https://github.com/PacktPublishing/Asynchronous-Programming-in-Python/tree/main/Chapter01). You don’t need anything special installed on your computer besides Python 3; if you need help with installation check the community instructions at https://wiki.python.org/moin/BeginnersGuide.

As in many aspects of life, programming requires clear objectives if success is to be achieved, and those objectives are usually formulated as objectively testable requirements. A set of requirements represents all the characteristics that a software solution must exhibit to be deemed satisfactory, i.e. the things that you must check to accept or reject a solution. They can include functional and non-functional aspects. Functional aspects are directly related to the product definition (‘If I do X, Y happens’) whereas non-functional requirements are not directly related to the solution per se but may be required for other reasons (e.g. ‘Implement using the Cloud to guarantee a certain level of availability’).

For example, in sports or board games there is usually the clear objective of winning a match, and it is usually easy to evaluate whether the player has achieved that objective or not. In basketball, you can see if the ball has passed through the hoop. The shot clock is an example of a non-functional requirement: it’s not necessary for scoring but is an important rule of the game nonetheless that must be complied with to avoid a penalty.

Synchronous programming: chess

A good way to learn how to think in a synchronous and structured way is to solve little chess puzzles. Chess is a ‘complete information’ game, which means that everybody involved in a game has complete awareness of the situation of the game. A chess puzzle is an individual practice mode in which the player must find a solution for an established game situation to finish the game (checkmate). Usually, chess puzzles have an optimal solution which is defined as the solution requiring the fewest moves to reach checkmate.

The following chess puzzle can be optimally solved by the white player in three moves:

Figure 1.2: A chess puzzle solvable in three moves by white

To solve this kind of problem the player must make their moves sequentially, taking into account the global state of the game (the positions of the pieces on the board), the value of each piece (for in chess each available piece type has a different value), and the potential reactions the opponent may make to the player’s moves. Remember that chess is a turn-based strategy game.

Many problems can be solved in this way, which we refer to as synchronous programming – the decomposition of steps into a cascade having a single line or thread of control. The execution of each step and time of execution are perfectly synchronized, and each step has full information about the global status, variables and available resources.

The following table shows the solution of the previous puzzle in three moves for white. Notice that the flow might change if conditions varied with the opponent’s moves (for example if in Step 1(b) the black player made a mistake):

Table 1.1: Solution for the chess puzzle

Asynchronous programming: soccer

A game like soccer is much more complex than chess. It involves two teams composed of 11 players each, and players are assigned positions (goalkeeper, defender, midfielder, forward) which impact their initial locations and potentially their ability to perform certain actions. The overall objective is to score (cause the ball to cross the opponent’s goal line). Any player can do this, and although at any given moment one team is defending and the other is attacking, the roles are fluid and continuous and ‘turns’ at shooting to goal are often unexpected.

The nature of the game allows for an infinite number of strategies. Usually, a team’s strategy involves not only retaining the ball but also making teammates run to distract the opposing team and to gradually occupy favorable ‘real estate’ on the field to improve the chances of any given shot.

The following three diagrams show a typical soccer play in which a defender takes control of the ball and after three moves scores a goal:

Figure 1.3: A soccer play starts with number 2 making a pass to number 10

The main execution timeline is always the one in which the ball is involved. Here it starts with player 2 taking control of the ball and making a pass to player 10, but once the pass is executed player 2 starts to run to a new position.

Figure 1.4: Second move: a dribble by number 10 and a run by number 2

At the second instant in time, multiple things are happening: player 10 dribbles past an opponent, while players 9, 2, and 11 move downfield for better positioning.

Figure 1.5: Third move: number 2 scores

At the third instant, player 10 waits until player 2 is in position to score, after which he passes the ball and player 2 is able to hit the net. The main execution line (scoring the goal) cannot be achieved if the supporting, parallel executions by multiple players are not completed.

In the same way, asynchronous programming is a technique in which some of an algorithm’s steps are executed in different lines of control than the main one, and those executions may occur simultaneously. Simultaneous execution is usually managed by the operating system or the programming language runtime, but simultaneous execution is not a requirement for asynchronous programming: asynchronous operations can also be run sequentially if desired.

It’s important to note that the multiple control lines in asynchronous programming don’t block each other. As in soccer, individual operations (the equivalent of players in soccer) are free to run unimpeded as other actions occur around them.

We have used the idea of control line in both examples without a formal definition. This is because there are several ways that modern computers control the execution of programs, depending on hardware characteristics, scheduling algorithms, memory management, and I/O handling. Moreover, programming languages and frameworks have their own approaches to concurrency that may vary depending on OS or hardware constraints.

In this section, we have introduced three key concepts which will be developed throughout the book: synchronous solutions, asynchronous solutions, and lines of control. Those concepts will be further elaborated in the specific context of computer science in the following section, to help you move from intuition and sports metaphors to real computer programming.

Central Processing Units (CPUs) function in a fetch – execute cycle. Specifically, the operating system (OS) fetches a set of instructions (program) from disk into memory, and they are then executed by the CPU. A program being executed is called a process. Loading a program into memory to become a process implies dividing memory into these sections:

• Text: This section of the memory allocated typically contains compiled code with a static set of instructions
• Data: Static data and global variables required for a running process
• Heap: Space reserved for dynamically allocated data structures (non-static, non-global variables)
• Stack: Local variables used in functions, which, if large enough, can compete with allocated heap space (causing a ‘stack overflow’ or ‘insufficient heap space’ error)

Although in an asynchronous program it may appear that all instructions in the set are being called at exactly the same time, technically each step is broken into blocks which are scheduled to be executed by the OS. Those blocks are executed so fast that it gives the impression that a processor is computing several things at the same time.

The change from execution of code blocks from one process to execution of blocks from another is a costly operation, called context switching. Context switching involves managing interruptions in the processing of a block, knowing the execution status of any given process, and waiting for other processes to complete, among other requirements for proper process flow.

Introducing threads

Modern computers typically have multiple cores, each of which is capable of executing a process. To better handle context switching, an abstraction was created: a thread, or atomic unit of processing. Each thread runs on a single core, and a processor can simultaneously run multiple threads from a single process by taking advantage of this architecture.

Threads are also called lightweight processes, since they must each individually conform to the structure described above for processes, but there is an important consideration: multiple threads of a process share the memory heap and code/data segments, which means that programmers must be careful to ensure that shared resource constraints are adhered to, but each thread maintains its own private stack.

The following diagram shows how processing can vary according to CPU and OS characteristics:

Figure 1.6: A single process/single thread processor on the left and a multithreaded processor on the right

What happens if a process has more threads than available cores? Thread context switching is ‘lighter’ because it involves saving and restoring less state, while process context switching is ‘heavier’ because it involves saving and restoring more state, including memory mappings. Therefore, in terms of efficiency, context switching between threads is generally faster and less resource-intensive than context switching between processes.

Some pieces of software are multiprocessor but not multithreaded, meaning that all processes are single-threaded (synchronous) but they can be split to take advantage of multiple processors.

There are two types of thread: kernel threads and user threads. User threads are created, managed, and bounded via the Application Programming Interface (API) provided by a system’s OS and managed by the individual program being run. The key point about user threads is that if one of them performs blocking operations, the entire process is blocked. This impacts the way multithreaded programs are designed.

The lifecycle of kernel threads, on the other hand, is entirely managed by the operating system. This type of thread has the advantage that if an operation blocks thread execution, the parent process is not blocked. Python’s default threading model is managed by the underlying operating system kernel, even if by default only one thread can run the interpreter at the time. We will explore this design in more detail in Anchor 4.

Processes, kernel threads and user threads are constructs that involve close management of the physical resources of a computer. As you might expect, modern programming languages provide abstractions to efficiently manage these concepts and the underlying resources. In the next section we will discuss three programming concepts central to multitasking: green threads, coroutines and fibers.

Just as user threads are overlaid on kernel threads via the OS API, green threads are implemented entirely within the runtime or virtual machine provided by the programming language. In Python, scheduling responsibility for green threads is part of the interpreter process that runs the threads.

The following table summarizes the most important differences between Python’s native threads and green threads:

Table 1.2: Characteristics of threads and green threads

Many programming languages have implemented green threads as their primary multitasking solution, but due to the limitations for multi-processing most have evolved to allow for cooperative multitasking through fibers and coroutines.

Fibers and coroutines

Fibers are like green threads in that they use a runtime scheduler that is independent of the underlying OS. However, instead of running until the scheduler interrupts their execution, fibers cooperate by ceding control to the next fiber in the same process. (Think of yarn being composed of multiple individual threads woven together.) This is also called cooperative multitasking.

A common drawback of fibers is that because scheduling control is passed to the developer, some fibers run or utilize resources over an extended period, reducing the resources available for execution of other fibers. Usually, fibers run inside a single thread.

The next step in the evolution of asynchronous processing is the coroutine, which is a function that can pause its own execution and later be resumed at the point at which it was interrupted. The following code starts the execution of a coroutine, then pauses its execution until some data is passed to resume the operation:

import datetime
def date_coroutine(_date:datetime.datetime):
    print(f"Your appointment is scheduled for {_date.strftime('%m/%d/%Y, %H:%M:%S')}")
    while True:
        current_date = (yield)
        if current_date > _date:
            print("Oops, your appointment already passed")
        else:
            print("You have time")
d1 = datetime.datetime(1981, 6, 29, 1, 0)
coroutine = date_coroutine(d1)
coroutine.__next__()
d2 = datetime.datetime(2018, 5, 3)
coroutine.send(d2)

The date_coroutine is initialized with d1, but it’s not executed until the __next__() method is invoked. It starts by printing the value of the argument, then waits until data is passed via the send() method. Notice that there are two points of access to the method, and run time may vary. Coroutines will be explored deeply in Anchor 2.

Multitasking features are a two-way street – you need to communicate with them to understand whether they have been executed if you want to trigger another dependent process. Callbacks, futures, and promises are ways to manage these flows.

Callbacks are functions that are passed as arguments to other functions or functions that are called inside other functions. These functions can be invoked when an event occurs. Callbacks are usually used as a join point for multithreading/multiprocessing solutions. The following example shows a toy example of a callback function that is invoked from each thread, after the thread has done some processing:

def worker(num, callback):
  print(f"Worker {num} starting...")
  time.sleep(num)  # Simulate some work
  print(f"Worker {num} finished.")
  callback(num)  # Call the callback
def callback_function(num):
  print(f"Callback for worker {num} called.")
if __name__ == "__main__":
  threads = []
  for i in range(5):
    thread = threading.Thread(target=worker, args=(i, callback_function))
    threads.append(thread)
    thread.start()
  for thread in threads:
    thread.join()

Multiprocessing and multithreading callbacks are treated in detail in Anchor 3. While very popular some time ago in other languages, such as JavaScript, this mechanism has some drawbacks that need to be mitigated, including nesting of callbacks, difficult debugging and race conditions. Other mechanisms have been developed to address these and other difficulties, like futures and promises.

The result of an asynchronous call is unknown at the start of the main thread’s execution, and the future/promise concept allows programmers to wait until something is returned before continuing a process. Futures/promises can be awaited up until their execution finishes and can execute a callback function when they end.

Semantics for futures/promises vary by programming language. In Python, futures are part of the standard language API, but promises are implemented by the community based on the Promises/A+ (https://promisesaplus.com/implementations) specification. Futures and promises are covered in detail in Anchor 2.

Besides the many concepts explored in this chapter, probably the three most important challenges that a programmer faces when designing an asynchronous solution are:

• Setting expectations: not all programming constructs are applicable in all contexts, and they come with costs. CPU- and I/O-bounded problems may not always benefit from multi-threaded approaches.
• Testing/debugging asynchronous code: testing is a crucial aspect of modern programming, and threads and coroutines can be complicated to debug. Some techniques and common patterns have been developed, and they will be discussed in later sections.
• Thread safety: shared resources always impose access management challenges. Concurrent changes to stored data are an obvious challenge, so it’s important to keep in mind key concepts like ACID compliance when designing database solutions. Likewise, shared resources (volatile/non-volatile memory, callback execution) must guarantee execution safety in multi-thread environments.

In this chapter we have informally introduced several key terms and concepts. In the next chapter we will go deep into actual Python constructs for multiprocessing and multithreading, including problems in which those techniques bring more value.

We learned in an intuitive way how to distinguish synchronous and asynchronous solutions to well-defined problems. Then we translated those ideas into standard computer science terminology. This will allow us to go deeper into the particulars of each concept as we focus on specific coding solutions. In Anchor 2 we will embark on the practical approach by coding and comparing the multiprocessing and multithreading solutions for a vanilla implementation of a CPU-intensive problem.