Michel Raynal

Concurrent Programming:
Algorithms, Principles,
and Foundations

123
Michel Raynal
Institut Universitaire de France
IRISA-ISTIC
Université de Rennes 1
Rennes Cedex
France

ISBN 978-3-642-32026-2 ISBN 978-3-642-32027-9 (eBook)

DOI 10.1007/978-3-642-32027-9
Springer Heidelberg New York Dordrecht London

Library of Congress Control Number: 2012944394

ACM Computing Classification (1998): F.1, D.1, B.3

 Springer-Verlag Berlin Heidelberg 2013

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or
information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the
work. Duplication of this publication or parts thereof is permitted only under the provisions of
the Copyright Law of the Publisher’s location, in its current version, and permission for use must always
be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright
Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
from the relevant protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Preface

As long as the grass grows and the rivers flow….

From American Indians

Homo sum: humani nihil a me alienum puto.

In Heautontimoroumenos, Publius Terencius (194–129 BC)

… Ce jour-là j’ai bien cru tenir quelque chose et que ma vie s’en trouverait changée.
Mais rien de cette nature n’est définitivement acquis.
Comme une eau, le monde vous traverse et pour un temps vous prête ses couleurs.
Puis se retire et vous replace devant ce vide qu’on porte en soi, devant cette espèce
d’insuffisance centrale de l’âme qu’il faut bien apprendre à côtoyer, à combattre,
et qui, paradoxalement, est peut-être notre moteur le plus sûr.
In L’usage du monde (1963), Nicolas Bouvier (1929–1998)

v
vi Preface

What synchronization is

A concurrent program is a program made up of several entities (processes, peers,

sensors, nodes, etc.) that cooperate to a common goal. This cooperation is made
possible thanks to objects shared by the entities. These objects are called con-
current objects. Let us observe that a concurrent object can be seen as abstracting a
service shared by clients (namely, the cooperating entities).

A fundamental issue of computing science and computing engineering consists

in the design and the implementation of concurrent objects. In order that concurrent
objects remain always consistent, the entities have to synchronize their accesses to
these objects. Ensuring correct synchronization among a set of cooperating entities
is far from being a trivial task. We are no longer in the world of sequential
programming, and the approaches and methods used in sequential computing are of
little help when one has to design concurrent programs. Concurrent programming
requires not only great care but also knowledge of its scientific foundations.
Moreover, concurrent programming becomes particularly difficult when one has to
cope with failures of cooperating entities or concurrent objects.

Why this book?

Since the early work of E.W. Dijkstra (1965), who introduced the mutual exclu-
sion problem, the concept of a process, the semaphore object, the notion of a
weakest precondition, and guarded commands (among many other contributions),
synchronization is no longer a catalog of tricks but a domain of computing science
with its own concepts, mechanisms, and techniques whose results can be applied in
many domains. This means that process synchronization has to be a major topic of
any computer science curriculum.

This book is on synchronization and the implementation of concurrent objects.

It presents in a uniform and comprehensive way the major results that have been
produced and investigated in the past 30 years and have proved to be useful from
both theoretical and practical points of view. The book has been written first for
people who are not familiar with the topic and the concepts that are presented.
These include mainly:
• Senior-level undergraduate students and graduate students in computer science
or computer engineering, and graduate students in mathematics who are inter-
ested in the foundations of process synchronization.
• Practitioners and engineers who want to be aware of the state-of-the-art concepts,
basic principles, mechanisms, and techniques encountered in concurrent
programming and in the design of concurrent objects suited to shared memory
systems.
Preface vii

Prerequisites for this book include undergraduate courses on algorithms and

base knowledge on operating systems. Selections of chapters for undergraduate
and graduate courses are suggested in the section titled ‘‘How to Use This Book’’
in the Afterword.

Content

As stressed by its title, this book is on algorithms, base principles, and foundations
of concurrent objects and synchronization in shared memory systems, i.e., systems
where the entities communicate by reading and writing a common memory. (Such
a corpus of knowledge is becoming more and more important with the advent of
new technologies such as multicore architectures.)

The book is composed of six parts. Three parts are more focused on base
synchronization mechanisms and the construction of concurrent objects, while the
other three parts are more focused on the foundations of synchronization. (A
noteworthy feature of the book is that nearly all the algorithms that are presented
are proved.)
• Part I is on lock-based synchronization, i.e., on well-known synchronization
concepts, techniques, and mechanisms. It defines the most important synchro-
nization problem in reliable asynchronous systems, namely the mutual exclusion
problem (Chap. 1). It then presents several base approaches which have been
proposed to solve it with machine-level instructions (Chap. 2). It also presents
traditional approaches which have been proposed at a higher abstraction level to
solve synchronization problems and implement concurrent objects, namely the
concept of a semaphore and, at an even more abstract level, the concepts of
monitor and path expression (Chap. 3).

• After the reader has become familiar with base concepts and mechanisms suited
to classical synchronization in reliable systems, Part II, which is made up of a
single chapter, addresses a fundamental concept of synchronization; namely, it
presents and investigates the concept of atomicity and its properties. This allows
for the formalization of the notion of a correct execution of a concurrent pro-
gram in which processes cooperate by accessing shared objects (Chap. 4).

• Part I has implicitly assumed that the cooperating processes do not fail. Hence,
the question: What does happen when cooperating entities fail? This is the main
issue addressed in Part III (and all the rest of the book); namely, it considers that
cooperating entities can halt prematurely (crash failure). To face the net effect of
asynchrony and failures, it introduces the notions of mutex-freedom and asso-
ciated progress conditions such as obstruction-freedom, non-blocking, and wait-
freedom (Chap. 5).
viii Preface

The rest of Part III focuses on hybrid concurrent objects (Chap. 6), wait-free
implementations of paradigmatic concurrent objects such as counters and store-
collect objects (Chap. 7), snapshot objects (Chap. 8), and renaming objects
(Chap. 9).

• Part IV, which is made up of a single chapter, is on software transactional

memory systems. This is a relatively new approach whose aim is to simplify the
job of programmers of concurrent applications. The idea is that programmers
have to focus their efforts on which parts of their multiprocess programs have to
be executed atomically and not on the way atomicity has to be realized (Chap. 10).

• Part V returns to the foundations side. It shows how reliable atomic read/write
registers (shared variables) can be built from non-atomic bits. This part consists
of three chapters. Chapter 11 introduces the notions of safe register, regular
register, and atomic register. Then, Chap. 12 shows how to build an atomic bit
from a safe bit. Finally, Chap. 13 shows how an atomic register of any size can
be built from safe and atomic bits.
This part shows that, while atomic read/write registers are easier to use than safe
read/write registers, they are not more powerful from a computability point-of-
view.

• Part VI, which concerns also the foundations side, is on the computational
power of concurrent objects. It is made up of four chapters. It first introduces the
notion of a consensus object and shows that consensus objects are universal
objects (Chap. 14). This means that, as soon as a system provides us with atomic
read/write registers and consensus objects, it is possible to implement in a wait-
free manner any object defined from a sequential specification.
Part VI then introduces the notion of self-implementation and shows how atomic
registers and consensus objects can be built from base objects of the same type
which are not reliable (Chap. 15). Then, it presents the notion of a consensus
number and the associated consensus hierarchy which allows the computability
power of concurrent objects to be ranked (Chap. 16). Finally, the last chapter of
the book focuses on the wait-free implementation of consensus objects from
read/write registers and failure detectors (Chap. 17).
To have a more complete feeling of the spirit of this book, the reader can also
consult the section ‘‘What Was the Aim of This Book’’ in the Afterword) which
describes what it is hoped has been learned from this book. Each chapter starts
with a short presentation of its content and a list of keywords; it terminates with a
summary of the main points that have explained and developed. Each of the six
parts of the book is also introduced by a brief description of its aim and its
technical content.
Preface ix

Acknowledgments

This book originates from lecture notes for undergraduate and graduate courses on
process synchronization that I give at the University of Rennes (France) and, as an
invited professor, at several universities all over the world. I would like to thank
the students for their questions that, in one way or another, have contributed to this
book.

I want to thank my colleagues Armando Castañeda (UNAM, MX), Ajoy Datta

(UNLV, Nevada), Achour Mostéfaoui (Université de Nantes), and François Taiani
(Lancaster University, UK) for their careful reading of chapters of this book.
Thanks also to François Bonnet (JAIST, Kanazawa), Eli Gafni (UCLA), Damien
Imbs (IRISA, Rennes), Segio Rajsbaum (UNAM, MX), Matthieu Roy (LAAS,
Toulouse), and Corentin Travers (LABRI, Bordeaux) for long discussions on wait-
freedom. Special thanks are due to Rachid Guerraoui (EPFL), with whom I dis-
cussed numerous topics presented in this book (and many other topics) during the
past seven years. I would also like to thank Ph. Louarn (IRISA, Rennes) who was
my Latex man when writing this book, and Ronan Nugent (Springer) for his
support and his help in putting it all together.

Last but not least (and maybe most importantly), I also want to thank all the
researchers whose results are presented in this book. Without their work, this book
would not exist (Since I typeset the entire text myself (– for the text and xfig
for figures–), any typesetting or technical errors that remain are my responsibility.)

Michel Raynal
Professeur des Universités
Institut Universitaire de France
IRISA-ISTIC, Université de Rennes 1
Campus de Beaulieu, 35042 Rennes, France

September–November 2005 and June–October 2011

Rennes, Mont-Louis (ARCHI’11), Gdańsk (SIROCCO’11), Saint-Philibert,
Hong Kong (PolyU), Macau, Roma (DISC’11), Tirana (NBIS’11),
Grenoble (SSS’11), Saint-Grégoire, Douelle, Mexico City (UNAM).
Contents

Part I Lock-Based Synchronization

1 The Mutual Exclusion Problem. . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1 Multiprocess Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.1 The Concept of a Sequential Process. . . . . . . . . . . . 3
1.1.2 The Concept of a Multiprocess Program . . . . . . . . . 4
1.2 Process Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 Processors and Processes . . . . . . . . . . . . . . . . . . . . 4
1.2.2 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.3 Synchronization: Competition. . . . . . . . . . . . . . . . . 5
1.2.4 Synchronization: Cooperation. . . . . . . . . . . . . . . . . 7
1.2.5 The Aim of Synchronization
Is to Preserve Invariants . . . . . . . . . . . . . . . . . . . . 7
1.3 The Mutual Exclusion Problem . . . . . . . . . . . . . . . . . . . . . 9
1.3.1 The Mutual Exclusion Problem (Mutex) . . . . . . . . . 9
1.3.2 Lock Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.3 Three Families of Solutions . . . . . . . . . . . . . . . . . . 12
1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.5 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2 Solving Mutual Exclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 15

2.1 Mutex Based on Atomic Read/Write Registers . . . . . . . . . .. 15
2.1.1 Atomic Register . . . . . . . . . . . . . . . . . . . . . . . . .. 15
2.1.2 Mutex for Two Processes:
An Incremental Construction . . . . . . . . . . . . . . . .. 17
2.1.3 A Two-Process Algorithm . . . . . . . . . . . . . . . . . .. 19
2.1.4 Mutex for n Processes:
Generalizing the Previous Two-Process Algorithm. .. 22
2.1.5 Mutex for n Processes:
A Tournament-Based Algorithm . . . . . . . . . . . . . .. 26
2.1.6 A Concurrency-Abortable Algorithm. . . . . . . . . . .. 29

xi
xii Contents

2.1.7 A Fast Mutex Algorithm . . . . . . . . . . . . . . . . . . . . 33

2.1.8 Mutual Exclusion in a Synchronous System . . . . . . . 37
2.2 Mutex Based on Specialized Hardware Primitives . . . . . . . . 38
2.2.1 Test&Set, Swap and Compare&Swap . . . . . . . . . . . 39
2.2.2 From Deadlock-Freedom to Starvation-Freedom . . . . 40
2.2.3 Fetch&Add . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.3 Mutex Without Atomicity . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.3.1 Safe, Regular and Atomic Registers . . . . . . . . . . . . 45
2.3.2 The Bakery Mutex Algorithm . . . . . . . . . . . . . . . . 48
2.3.3 A Bounded Mutex Algorithm . . . . . . . . . . . . . . . . . 53
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.5 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.6 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

3 Lock-Based Concurrent Objects . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.1 Concurrent Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.1.1 Concurrent Object. . . . . . . . . . . . . . . . . . . . . . . . . 61
3.1.2 Lock-Based Implementation . . . . . . . . . . . . . . . . . . 62
3.2 A Base Synchronization Object: the Semaphore . . . . . . . . . . 63
3.2.1 The Concept of a Semaphore . . . . . . . . . . . . . . . . . 63
3.2.2 Using Semaphores to Solve
the Producer-Consumer Problem. . . . . . . . . ...... 65
3.2.3 Using Semaphores to Solve
a Priority Scheduling Problem . . . . . . . . . . ...... 71
3.2.4 Using Semaphores to Solve
the Readers-Writers Problem . . . . . . . . . . . ...... 74
3.2.5 Using a Buffer to Reduce Delays
for Readers and Writers. . . . . . . . . . . . . . . . . . . . . 78
3.3 A Construct for Imperative Languages: the Monitor . . . . . . . 81
3.3.1 The Concept of a Monitor . . . . . . . . . . . . . . . . . . . 82
3.3.2 A Rendezvous Monitor Object . . . . . . . . . . . . . . . . 83
3.3.3 Monitors and Predicates. . . . . . . . . . . . . . . . . . . . . 85
3.3.4 Implementing a Monitor from Semaphores . . . . . . . 87
3.3.5 Monitors for the Readers-Writers Problem . . . . . . . . 89
3.3.6 Scheduled Wait Operation . . . . . . . . . . . . . . . . . . . 94
3.4 Declarative Synchronization: Path Expressions . . . . . . . . . . . 95
3.4.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.4.2 Using Path Expressions to Solve
Synchronization Problems . . . . . . . . . . . . . ...... 97
3.4.3 A Semaphore-Based Implementation
of Path Expressions. . . . . . . . . . . . . . . . . . . . . . . . 98
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.6 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.7 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Contents xiii

Part II On the Foundations Side: The Atomicity Concept

4 Atomicity: Formal Definition and Properties . . . . . . . . . . . . . . . . 113

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.2 Computation Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.2.1 Processes and Operations. . . . . . . . . . . . . . . . . . . . 115
4.2.2 Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.2.3 Histories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.2.4 Sequential History. . . . . . . . . . . . . . . . . . . . . . . . . 119
4.3 Atomicity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.3.1 Legal History . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.3.2 The Case of Complete Histories . . . . . . . . . . . . . . . 121
4.3.3 The Case of Partial Histories . . . . . . . . . . . . . . . . . 123
4.4 Object Composability and Guaranteed
Termination Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.4.1 Atomic Objects Compose for Free . . . . . . . . . . . . . 125
4.4.2 Guaranteed Termination . . . . . . . . . . . . . . . . . . . . 127
4.5 Alternatives to Atomicity. . . . . . . . . . . . . . . . . . . . . . . . . . 128
4.5.1 Sequential Consistency . . . . . . . . . . . . . . . . . . . . . 128
4.5.2 Serializability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Part III Mutex-Free Synchronization

5 Mutex-Free Concurrent Objects . . . . . . . . . . . . . . . . . . . . . . . . . 135

5.1 Mutex-Freedom and Progress Conditions . . . . . . . . . . . . . . . 135
5.1.1 The Mutex-Freedom Notion . . . . . . . . . . . . . . . . . . 135
5.1.2 Progress Conditions . . . . . . . . . . . . . . . . . . . . . . . 137
5.1.3 Non-blocking with Respect to Wait-Freedom . . . . . . 140
5.2 Mutex-Free Concurrent Objects . . . . . . . . . . . . . . . . . . . . . 140
5.2.1 The Splitter: A Simple Wait-Free Object from
Read/Write Registers. . . . . . . . . . . . . . . . . . . . . .. 140
5.2.2 A Simple Obstruction-Free Object from
Read/Write Registers. . . . . . . . . . . . . . . . . . . . . .. 143
5.2.3 A Remark on Compare&Swap: The ABA Problem. .. 145
5.2.4 A Non-blocking Queue Based on
Read/Write Registers and Compare&Swap . . . . . .. 146
5.2.5 A Non-blocking Stack Based on
Compare&Swap Registers . . . . . . . . . . . . . . . . . .. 150
5.2.6 A Wait-Free Stack Based on
Fetch&Add and Swap Registers . . . . . . . . . . . . . .. 152
xiv Contents

5.3 Boosting Obstruction-Freedom to Stronger Progress

in the Read/Write Model . . . . . . . . . . . . . . . . . . . . . . . . . . 155
5.3.1 Failure Detectors . . . . . . . . . . . . . . . . . . . . . . . . . 155
5.3.2 Contention Managers for Obstruction-Free
Object Implementations . . . . . . . . . . . . . . . . . . . . . 157
5.3.3 Boosting Obstruction-Freedom to Non-blocking . . . . 158
5.3.4 Boosting Obstruction-Freedom to Wait-Freedom . . . 159
5.3.5 Mutex-Freedom Versus Loops Inside a Contention
Manager Operation . . . . . . . . . . . . . . . . . . . . . . . . 161
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
5.5 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
5.6 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

6 Hybrid Concurrent Objects . . . . . . . . . . . . . . . . . . . . . . . . . ... 165

6.1 The Notion of a Hybrid Implementation . . . . . . . . . . . . ... 165
6.1.1 Lock-Based Versus Mutex-Free Operation:
Static Hybrid Implementation. . . . . . . . . . . . . . ... 166
6.1.2 Contention Sensitive (or Dynamic Hybrid)
Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . 166
6.1.3 The Case of Process Crashes . . . . . . . . . . . . . . . . . 166
6.2 A Static Hybrid Implementation of a Concurrent Set Object . . . 167
6.2.1 Definition and Assumptions . . . . . . . . . . . . . . . . . . 167
6.2.2 Internal Representation and Operation
Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6.2.3 Properties of the Implementation . . . . . . . . . . . . . . 171
6.3 Contention-Sensitive Implementations . . . . . . . . . . . . . . . . . 172
6.3.1 Contention-Sensitive Binary Consensus . . . . . . . . . . 172
6.3.2 A Contention Sensitive Non-blocking
Double-Ended Queue . . . . . . . . . . . . . . . . . . . ... 176
6.4 The Notion of an Abortable Object . . . . . . . . . . . . . . . . ... 181
6.4.1 Concurrency-Abortable Object . . . . . . . . . . . . . ... 181
6.4.2 From a Non-blocking Abortable Object
to a Starvation-Free Object . . . . . . . . . . . . . . . . . . 183
6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
6.6 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
6.7 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

7 Wait-Free Objects from Read/Write Registers Only . . . . . . . . . . 189

7.1 A Wait-Free Weak Counter for Infinitely Many Processes. . . 189
7.1.1 A Simple Counter Object. . . . . . . . . . . . . . . . . . . . 190
7.1.2 Weak Counter Object for Infinitely Many Processes. . . 191
7.1.3 A One-Shot Weak Counter Wait-Free Algorithm . . . 193
7.1.4 Proof of the One-Shot Implementation . . . . . . . . . . 194
7.1.5 A Multi-Shot Weak Counter Wait-Free Algorithm . . . 199
Contents xv

7.2 Store-Collect Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

7.2.1 Store-Collect Object: Definition . . . . . . . . . . . . . . . 201
7.2.2 An Adaptive Store-Collect Implementation . . . . . . . 204
7.2.3 Proof and Cost of the Adaptive Implementation . . . . 208
7.3 Fast Store-Collect Object . . . . . . . . . . . . . . . . . . . . . . . . . . 211
7.3.1 Fast Store-Collect Object: Definition. . . . . . . . . . . . 211
7.3.2 A Fast Algorithm for the store collectðÞ Operation . . . 212
7.3.3 Proof of the Fast Store-Collect Algorithm . . . . . . . . 215
7.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
7.5 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
7.6 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

8 Snapshot Objects from Read/Write Registers Only . . . . . . . . . . . 219

8.1 Snapshot Objects: Definition . . . . . . . . . . . . . . . . . . . . . . . 219
8.2 Single-Writer Snapshot Object . . . . . . . . . . . . . . . . . . . . . . 220
8.2.1 An Obstruction-Free Implementation. . . . . . . . . . . . 221
8.2.2 From Obstruction-Freedom to Bounded
Wait-Freedom . . . . . . . . . . . . . . . . . . . . . . . . . .. 223
8.2.3 One-Shot Single-Writer Snapshot Object:
Containment Property . . . . . . . . . . . . . . . . . . . . .. 227
8.3 Single-Writer Snapshot Object with Infinitely
Many Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 228
8.4 Multi-Writer Snapshot Object. . . . . . . . . . . . . . . . . . . . . .. 230
8.4.1 The Strong Freshness Property . . . . . . . . . . . . . . .. 231
8.4.2 An Implementation of a Multi-Writer
Snapshot Object . . . . . . . . . . . . . . . . . . . . . . . . . . 231
8.4.3 Proof of the Implementation. . . . . . . . . . . . . . . . . . 234
8.5 Immediate Snapshot Objects . . . . . . . . . . . . . . . . . . . . . . . 238
8.5.1 One-Shot Immediate Snapshot Object: Definition . . . 238
8.5.2 One-Shot Immediate Snapshot Versus
One-Shot Snapshot . . . . . . . . . . . . . . . . . . . . . . .. 238
8.5.3 An Implementation of One-Shot
Immediate Snapshot Objects . . . . . . . . . . . . . . . .. 240
8.5.4 A Recursive Implementation of a One-Shot
Immediate Snapshot Object . . . . . . . . . . . . . . . . . . 244
8.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
8.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
8.8 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
xvi Contents

9 Renaming Objects from Read/Write Registers Only . . . . . . . . . . 249

9.1 Renaming Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
9.1.1 The Base Renaming Problem . . . . . . . . . . . . . . . . . 249
9.1.2 One-Shot Renaming Object . . . . . . . . . . . . . . . . . . 250
9.1.3 Adaptive Implementations . . . . . . . . . . . . . . . . . . . 250
9.1.4 A Fundamental Result . . . . . . . . . . . . . . . . . . . . . . 251
9.1.5 Long-Lived Renaming. . . . . . . . . . . . . . . . . . . . . . 252
9.2 Non-triviality of the Renaming Problem . . . . . . . . . . . . . . . 252
9.3 A Splitter-Based Optimal Time-Adaptive Implementation . . . 254
9.4 A Snapshot-Based Optimal Size-Adaptive Implementation. . . 256
9.4.1 A Snapshot-Based Implementation . . . . . . . . . . . . . 256
9.4.2 Proof of the Implementation. . . . . . . . . . . . . . . . . . 258
9.5 Recursive Store-Collect-Based Size-Adaptive
Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
9.5.1 A Recursive Renaming Algorithm . . . . . . . . . . . . . 259
9.5.2 An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
9.5.3 Proof of the Renaming Implementation . . . . . . . . . . 263
9.6 Variant of the Previous Recursion-Based
Renaming Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 266
9.6.1 A Renaming Implementation Based
on Immediate Snapshot Objects . . . . . . . . . . . . . .. 266
9.6.2 An Example of a Renaming Execution . . . . . . . . .. 268
9.7 Long-Lived Perfect Renaming Based
on Test&Set Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
9.7.1 Perfect Adaptive Renaming . . . . . . . . . . . . . . . . . . 269
9.7.2 Perfect Long-Lived Test&Set-Based Renaming . . . . 270
9.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
9.9 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
9.10 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

Part IV The Transactional Memory Approach

10 Transactional Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

10.1 What Are Software Transactional Memories . . . . . . . . . . . . 277
10.1.1 Transactions = High-Level Synchronization . . . . . . . 277
10.1.2 At the Programming Level. . . . . . . . . . . . . . . . . . . 279
10.2 STM System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
10.2.1 Speculative Executions, Commit
and Abort of a Transaction . . . . . . . . . . . . . . . . . . 281
10.2.2 An STM Consistency Condition: Opacity . . . . . . . . 282
10.2.3 An STM Interface. . . . . . . . . . . . . . . . . . . . . . . . . 282
10.2.4 Incremental Reads and Deferred Updates. . . . . . . . . 283
Contents xvii

10.2.5 Read-Only Versus Update Transactions . . . . . . . ... 283

10.2.6 Read Invisibility . . . . . . . . . . . . . . . . . . . . . . . ... 284
10.3 A Logical Clock-Based STM System: TL2 . . . . . . . . . . ... 284
10.3.1 Underlying System and Control Variables
of the STM System . . . . . . . . . . . . . . . . . . . . . ... 284
10.3.2 Underlying Principle: Consistency
with Respect to Transaction Birth Date . . . . . . . . . . 285
10.3.3 The Implementation of an Update Transaction . . . . . 286
10.3.4 The Implementation of a Read-Only Transaction . . . 288
10.4 A Version-Based STM System: JVSTM . . . . . . . . . . . . . . . 289
10.4.1 Underlying and Control Variables
of the STM System . . . . . . . . . . . . . . . . . . . . . . . . 290
10.4.2 The Implementation of an Update Transaction . . . . . 291
10.4.3 The Implementation of a Read-Only Transaction . . . 293
10.5 A Vector Clock-Based STM System . . . . . . . . . . . . . . . . . . 293
10.5.1 The Virtual World Consistency Condition . . . . . . . . 293
10.5.2 An STM System for Virtual World Consistency . . . . 295
10.5.3 The Algorithms Implementing
the STM Operations . . . . . . . . . . . . . . . . . . . . . . . 296
10.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
10.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
10.8 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

Part V On the Foundations Side:

From Safe Bits to Atomic Registers

11 Safe, Regular, and Atomic Read/Write Registers . . . . . . . . . . . . . 305

11.1 Safe, Regular, and Atomic Registers . . . . . . . . . . . . . . . . . . 305
11.1.1 Reminder: The Many Faces of a Register . . . . . . . . 305
11.1.2 From Regularity to Atomicity: A Theorem . . . . . . . 308
11.1.3 A Fundamental Problem:
The Construction of Registers . . . . . . . . . . . . . . .. 310
11.2 Two Very Simple Bounded Constructions . . . . . . . . . . . . .. 311
11.2.1 Safe/Regular Registers:
From Single-Reader to Multi-Reader. . . . . . . . . . .. 311
11.2.2 Binary Multi-Reader Registers:
From Safe to Regular . . . . . . . . . . . . . . . . . . . . . . 313
11.3 From Bits to b-Valued Registers. . . . . . . . . . . . . . . . . . . . . 314
11.3.1 From Safe Bits to b-Valued Safe Registers . . . . . . . 314
11.3.2 From Regular Bits to Regular b-Valued Registers. . . 315
11.3.3 From Atomic Bits to Atomic b-Valued Registers . . . 319
xviii Contents

11.4 Three Unbounded Constructions . . . . . . . . . . ........... 321

11.4.1 SWSR Registers:
From Unbounded Regular to Atomic. ........... 322
11.4.2 Atomic Registers:
From Unbounded SWSR to SWMR . ........... 324
11.4.3 Atomic Registers:
From Unbounded SWMR to MWMR ........... 325
11.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . ........... 327
11.6 Bibliographic Notes . . . . . . . . . . . . . . . . . . ........... 327

12 From Safe Bits to Atomic Bits:

Lower Bound and Optimal Construction . . . . . . . . . . . . . . . . . . . 329
12.1 A Lower Bound Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 329
12.1.1 Two Preliminary Lemmas . . . . . . . . . . . . . . . . . . . 330
12.1.2 The Lower Bound Theorem . . . . . . . . . . . . . . . . . . 331
12.2 A Construction of an Atomic Bit from Three Safe Bits . . . . . 334
12.2.1 Base Architecture of the Construction . . . . . . . . . . . 334
12.2.2 Underlying Principle and Signaling Scheme. . . . . . . 335
12.2.3 The Algorithm Implementing
the Operation R:writeðÞ . . . . . . . . . . . . . . . . . .... 336
12.2.4 The Algorithm Implementing
the Operation R:readðÞ . . . . . . . . . . . . . . . . . . . . . 336
12.2.5 Cost of the Construction . . . . . . . . . . . . . . . . . . . . 338
12.3 Proof of the Construction of an Atomic Bit . . . . . . . . . . . . . 338
12.3.1 A Preliminary Theorem . . . . . . . . . . . . . . . . . . . . . 338
12.3.2 Proof of the Construction. . . . . . . . . . . . . . . . . . . . 340
12.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
12.5 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
12.6 Exercise. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

13 Bounded Constructions of Atomic b-Valued Registers . . . . . . ... 347

13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 347
13.2 A Collision-Free (Pure Buffers) Construction . . . . . . . . . ... 349
13.2.1 Internal Representation of the Atomic b-Valued
Register R . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 349
13.2.2 Underlying Principle: Two-Level Switch
to Ensure Collision-Free Accesses to Buffers . . . ... 349
13.2.3 The Algorithms Implementing
the Operations R:writeðÞ and R:readðÞ . . . . . . . . . . 350
13.2.4 Proof of the Construction: Collision-Freedom. . . . . . 352
13.2.5 Correctness Proof . . . . . . . . . . . . . . . . . . . . . . . . . 355
13.3 A Construction Based on Impure Buffers . . . . . . . . . . . . . . 357
13.3.1 Internal Representation of the Atomic b-Valued
Register R . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 357
Contents xix

13.3.2 An Incremental Construction . . . . . . . .......... 358

13.3.3 The Algorithms Implementing
the Operations R:writeðÞ and R:readðÞ .......... 360
13.3.4 Proof of the Construction. . . . . . . . . . .......... 360
13.3.5 From SWSR to SWMR b-Valued
Atomic Register . . . . . . . . . . . . . . . . .......... 367
13.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . .......... 368
13.5 Bibliographic Notes . . . . . . . . . . . . . . . . . . . .......... 368

Part VI On the Foundations Side:

The Computability Power of Concurrent Objects (Consensus)

14 Universality of Consensus . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 371

14.1 Universal Object, Universal Construction,
and Consensus Object . . . . . . . . . . . . . . . . . . . . . . . . . ... 371
14.1.1 Universal (Synchronization) Object
and Universal Construction . . . . . . . . . . . . . . . . . . 371
14.1.2 The Notion of a Consensus Object . . . . . . . . . . . . . 372
14.2 Inputs and Base Principles of Universal Constructions . . . . . 373
14.2.1 The Specification of the Constructed Object . . . . . . 373
14.2.2 Base Principles of Universal Constructions . . . . . . . 374
14.3 An Unbounded Wait-Free Universal Construction. . . . . . . . . 374
14.3.1 Principles and Description of the Construction . . . . . 375
14.3.2 Proof of the Construction. . . . . . . . . . . . . . . . . . . . 378
14.3.3 Non-deterministic Objects . . . . . . . . . . . . . . . . . . . 382
14.3.4 Wait-Freedom Versus Bounded Wait-Freedom . . . . . 383
14.4 A Bounded Wait-Free Universal Construction . . . . . . . . . . . 384
14.4.1 Principles of the Construction . . . . . . . . . . . . . . . . 384
14.4.2 Proof of the Construction. . . . . . . . . . . . . . . . . . . . 388
14.4.3 Non-deterministic Objects . . . . . . . . . . . . . . . . . . . 391
14.5 From Binary Consensus to Multi-Valued Consensus . . . . . . . 391
14.5.1 A Construction Based on the Bit Representation
of Proposed Values . . . . . . . . . . . . . . . . . . . . . . . . 392
14.5.2 A Construction for Unbounded Proposed Values . . . 394
14.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
14.7 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
14.8 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

15 The Case of Unreliable Base Objects. . . . . . . . . . . . . . . . . ..... 399

15.1 Responsive Versus Non-responsive Crash Failures . . . ..... 400
15.2 SWSR Registers Prone to Crash Failures . . . . . . . . . . ..... 400
15.2.1 Reliable Register When Crash Failures
Are Responsive: An Unbounded Construction ..... 401
xx Contents

15.2.2 Reliable Register When Crash Failures Are

Responsive: A Bounded Construction . . . . . . . . . . . 403
15.2.3 Reliable Register When Crash Failures Are Not
Responsive: An Unbounded Construction . . . . . . . . 406
15.3 Consensus When Crash Failures Are Responsive:
A Bounded Construction . . . . . . . . . . . . . . . . . . . . . . . . . . 408
15.3.1 The ‘‘Parallel Invocation’’ Approach
Does Not Work . . . . . . . . . . . . . . . . . . . . . . . . . . 408
15.3.2 A t-Tolerant Wait-Free Construction . . . . . . . . . . . . 409
15.3.3 Consensus When Crash Failures Are Not Responsive:
An Impossibility . . . . . . . . . . . . . . . . . . . . . . . . . . 410
15.4 Omission and Arbitrary Failures . . . . . . . . . . . . . . . . . . . . . 410
15.4.1 Object Failure Modes . . . . . . . . . . . . . . . . . . . . . . 410
15.4.2 Simple Examples . . . . . . . . . . . . . . . . . . . . . . . . . 412
15.4.3 Graceful Degradation . . . . . . . . . . . . . . . . . . . . . . 413
15.4.4 Fault-Tolerance Versus Graceful Degradation . . . . . 417
15.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
15.6 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
15.7 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

16 Consensus Numbers and the Consensus Hierarchy . . . . . . . . . . . 421

16.1 The Consensus Number Notion . . . . . . . . . . . . . . . . . . . . . 421
16.2 Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
16.2.1 Schedule, Configuration, and Valence . . . . . . . . . . . 422
16.2.2 Bivalent Initial Configuration . . . . . . . . . . . . . . . . . 423
16.3 The Weak Wait-Free Power of Atomic Registers . . . . . . . . . 425
16.3.1 The Consensus Number of Atomic
Read/Write Registers Is 1 . . . . . . . . . . . . . . . . . . . 425
16.3.2 The Wait-Free Limit of Atomic Registers . . . . . . . . 428
16.4 Objects Whose Consensus Number Is 2. . . . . . . . . . . . . . . . 429
16.4.1 Consensus from Test&Set Objects . . . . . . . . . . . . . 429
16.4.2 Consensus from Queue Objects . . . . . . . . . . . . . . . 431
16.4.3 Consensus from Swap Objects . . . . . . . . . . . . . . . . 432
16.4.4 Other Objects for Wait-Free Consensus
in a System of Two Processes . . . . . . . . . . . . . . . . 432
16.4.5 Power and Limit of the Previous Objects. . . . . . . . . 433
16.5 Objects Whose Consensus Number Is þ1 . . . . . . . . . . . . . 438
16.5.1 Consensus from Compare&Swap Objects . . . . . . . . 439
16.5.2 Consensus from Mem-to-Mem-Swap Objects . . . . . . 440
16.5.3 Consensus from an Augmented Queue . . . . . . . . . . 442
16.5.4 From a Sticky Bit to Binary Consensus . . . . . . . . . . 442
16.5.5 Impossibility Result . . . . . . . . . . . . . . . . . . . . . . . 443
Contents xxi

16.6 Hierarchy of Atomic Objects . . . . . . ......... . . . . . . . . 443

16.6.1 From Consensus Numbers to a Hierarchy . . . . . . . . 443
16.6.2 On Fault Masking . . . . . . . . ......... . . . . . . . . 444
16.6.3 Robustness of the Hierarchy. ......... . . . . . . . . 445
16.7 Summary . . . . . . . . . . . . . . . . . . . . ......... . . . . . . . . 445
16.8 Bibliographic Notes . . . . . . . . . . . . ......... . . . . . . . . 445
16.9 Exercises and Problems . . . . . . . . . . ......... . . . . . . . . 446

17 The Alpha(s) and Omega of Consensus:

Failure Detector-Based Consensus. . . . . . . . . . . . . . . . . . . . . . . . 449
17.1 De-constructing Compare&Swap . . . . . . . . . . . . . . . . . . . . 450
17.2 A Liveness-Oriented Abstraction: The Failure Detector X . . . 452
17.2.1 Definition of X. . . . . . . . . . . . . . . . . . . . . . . . . . . 452
17.2.2 X-Based Consensus:
X as a Resource Allocator or a Scheduler . . . . . . .. 453
17.3 Three Safety-Oriented Abstractions:
Alpha1 , Alpha2 , and Alpha3 . . . . . . . . . . . . . . . . . . . . . . . . 454
17.3.1 A Round-Free Abstraction: Alpha1 . . . . . . . . . . . . . 454
17.3.2 A Round-Based Abstraction: Alpha2 . . . . . . . . . . . . 455
17.3.3 Another Round-Free Abstraction: Alpha3 . . . . . . . . . 456
17.3.4 The Rounds Seen as a Resource . . . . . . . . . . . . . . . 457
17.4 X-Based Consensus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
17.4.1 Consensus from Alpha1 Objects and X . . . . . . . . . . 457
17.4.2 Consensus from an Alpha2 Object and X. . . . . . . . . 459
17.4.3 Consensus from an Alpha3 Object and X. . . . . . . . . 460
17.4.4 When the Eventual Leader Elected by X
Does Not Participate . . . . . . . . . . . . . . . . . . . . . .. 463
17.4.5 The Notion of an Indulgent Algorithm . . . . . . . . .. 464
17.4.6 Consensus Object Versus X . . . . . . . . . . . . . . . . .. 464
17.5 Wait-Free Implementations of the Alpha1 and Alpha2
Abstractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 465
17.5.1 Alpha1 from Atomic Registers . . . . . . . . . . . . . . .. 465
17.5.2 Alpha2 from Regular Registers . . . . . . . . . . . . . . .. 467
17.6 Wait-Free Implementations of the Alpha2 Abstraction
from Shared Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472
17.6.1 Alpha2 from Unreliable Read/Write Disks . . . . . . . . 472
17.6.2 Alpha2 from Active Disks . . . . . . . . . . . . . . . . . . . 476
17.7 Implementing X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
17.7.1 The Additional Timing Assumption EWB . . . . . . . . 478
17.7.2 An EWB-Based Implementation of X . . . . . . . . . . . 479
17.7.3 Proof of the Construction. . . . . . . . . . . . . . . . . . . . 481
17.7.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
xxii Contents

17.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

17.9 Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485
17.10 Exercises and Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 486

Afterword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
Afterword

What Was the Aim of This Book

The practice of sequential computing has greatly benefited from the results of the
theory of sequential computing that were captured in the study of formal languages
and automata theory. Everyone knows what can be computed (computability) and
what can be computed efficiently (complexity). All these results constitute the
foundations of sequential computing, which, thanks to them, has become a science.
These theoretical results and algorithmic principles have been described in a lot of
books from which students can learn basic results, algorithms, and principles of
sequential computing (e.g., [79, 85, 114, 151, 176, 203, 211, 258] to cite a few).
Synchronization is coming back, but is it the same? While books do exist for
traditional synchronization (e.g., [27, 58]), very few books present in a
comprehensive way the important results that have been discovered in the past
20 years.1 Hence, even if it describes a lot of algorithms implementing concurrent
objects, the aim of this book is not to be a catalog of algorithms. Its ambition is not
only to present synchronization algorithms but also to introduce the reader to the
theory that underlies the implementation of concurrent objects in the presence of
asynchrony and process crashes.

To summarize, thanks to appropriate curricula (and good associated books),

students now have a good background in the theory and practice of sequential
computing. An aim of this book is to try to provide them with an equivalent
background when they have to face the net effect of asynchrony and failures in the
context of shared memory systems.2 Hence, all the concepts, principles, and
mechanisms presented in the book aim at attaining this target.

1
The book by Gadi Taubenfeld [262] and the book by Maurice Herlihy and Nir Shavit [146] are
two such books.
2
For message-passing systems, the reader can consult [40, 60, 115, 176, 201, 236, 237, 248].

M. Raynal, Concurrent Programming: Algorithms, Principles, and Foundations, 489

DOI: 10.1007/978-3-642-32027-9,  Springer-Verlag Berlin Heidelberg 2013
490 Afterword

Technology is what makes everyday life easier. Science is what allows us to

transcend it, and capture the deep nature of the objects we are manipulating.
To that end, it provides us with the right concepts to master and understand what
we are doing. When considering synchronization and concurrent objects
encountered in computing science, an ambition of this book is to be a step in
this direction.

Most Important Concepts, Notions, and Mechanisms Presented

in This Book

Chapter 1: Competition, concurrent object, cooperation, deadlock-freedom,

invariant, liveness, lock object, multiprocess program, mutual exclusion, safety,
sequential process, starvation-freedom, synchronization.
Chapter 2: Atomic read/write register, lock object, mutual exclusion, safe read/
write register, specialized hardware primitive (test&set, fetch&add,
compare&swap).
Chapter 3: Declarative synchronization, imperative synchronization, lock-based
implementation, monitor, path expression, predicate transfer, semaphore.
Chapter 4: Atomicity, legal history, history, linearizability, locality property,
partial operation, sequential consistency, sequential history, serializability, total
operation.
Chapter 5: Contention manager, implementation boosting, mutex-freedom,
obstruction-freedom, non-blocking, process crash, progress condition, wait-
freedom.
Chapter 6: Abortable object, binary consensus, concurrent set object, contention-
sensitive implementation, double-ended queue, hybrid (static versus dynamic)
implementation, LL/SC primitive operations, linearization point, non-blocking to
starvation-freedom.
Chapter 7: Adaptive implementation, fast store-collect, favorable circumstances,
infinitely many processes, store-collect object, weak counter.
Chapter 8: Atomic snapshot object, immediate snapshot object, infinitely many
processes, one-shot object, read/write system, recursive algorithm.
Chapter 9: Adaptive algorithm, immediate snapshot object, grid of splitters,
recursive algorithm, renaming object (one-shot versus long-lived).
Chapter 10: Abort, atomic execution unit, commit, deferred updates, incremental
read (snapshot), lock, multi-version, opacity, read invisibility, read/modify/write,
software transactional memory, speculative execution, transaction (read-only,
write-only, update), virtual world consistency.
Afterword 491

Chapter 11: Atomic/regular/safe register, binary versus multi-valued register,

bounded versus unbounded construction, register construction, sequence number,
SWSR/SWMR/MWSR/MWMR, single-reader versus multi-reader, single-writer
versus multi-writer, timestamp.
Chapter 12: Atomic bit, bounded construction, linearizability, safe bit, switch,
wait-free construction.
Chapter 13: Atomic bit, atomic register construction, collision-freedom, finite state
automaton, linearizability, pure/impure buffer, safe buffer, switch, SWSR register,
wait-freedom.
Chapter 14: Atomic object, binary consensus, bounded construction, consensus
object, deterministic versus non-deterministic object, linearizability, multi-valued
consensus, sequential specification, total operation, total order, universal con-
struction, universal object, wait-freedom.
Chapter 15: Arbitrary failure, atomic read/write register, bounded versus
unbounded implementation, consensus object, crash failure, graceful degradation,
omission failure, responsive versus non-responsive failure, self-
implementation, t-tolerant implementation, wait-freedom.
Chapter 16: Atomic object, bivalent configuration, consensus object, consensus
number, consensus hierarchy, indistinguishability, monovalent configuration,
valence, wait-freedom.
Chapter 17: Active disk, adopt-commit, alpha abstraction, atomic register, com-
pare&swap, consensus object, indulgence, omega abstraction, regular register,
round-based abstraction, shared disk, store-collect, timing assumption.

How to Use This Book

This section presents courses on synchronization and concurrent objects which can
benefit from the concepts, algorithms and principles presented in this book. The
first course consists of one semester course for the last year of the undergraduate
level, while the two one-semester courses are more appropriate for the graduate
level.
• Undergraduate students.
A one-semester course could first focus on Part I devoted to mutual exclusion.
(The reader can notice this part of the book has plenty of exercises.)
Then, the course could address (a) the underlying theory, namely Part II devoted
to the formalization of the atomicity concept (in order for students to have a
clear view of the foundations on what they have learned), and (b) the notion of
mutex-freedom and associated progress conditions introduced in Chap. 5.
Examples taken from Chaps. 7–9 can be used to illustrate these notions.
492 Afterword

Finally, Part IV devoted to software transactional memory can be used to

introduce students to a new approach that addresses the design of multiprocess
programs.
• Graduate students.
Assuming that the content of the one-semester course for undergraduate students
is known and mastered, a first one-semester course for graduate students could
first study the notion of hybrid implementation (Chap. 6) and then investigate
the power of read/write registers to wait-free implement concurrent objects. This
is what is addressed in detail in Chaps. 5–9.
Then, the course could survey Part V which describes various constructions of
atomic read/write registers from safe registers. A selection of algorithms pre-
sented in Chaps. 11–13 can be used to that end.
A second one-semester course should be devoted to computability in the pres-
ence of failures, which is the topic addressed in Part VI of the book. The notions
of a universal construction (Chap. 14), its reliability (Chap. 15), the consensus
number hierarchy (Chap. 16), and the construction of consensus objects
(Chap. 17) are fundamental notions to master both the practice and the theory of
synchronization in the presence of failures.
Of course, this book can also be used by engineers and researchers who work on
shared memory multiprocessor systems, or multicore architectures, and are
interested in (a) what can be done in the presence of asynchrony and failures, and
(b) how it has to be done.

A Look at Other Books

This section briefly presents a few other books which address synchronization in
shared memory systems.
• Books on message-passing and shared memory:
– The book by H. Attiya and J. Welch [41], the book by A. Kshemkalyani and
M. Singhal [181], and the book by N. Lynch [201] consider both the
message-passing model and the shared memory model.
More precisely, Part IIA of Lynch’s book is devoted to the asynchronous
shared memory model. The algorithms are described in the input/output
automata model and formally proved using this formalism. In addition to
mutual exclusion, this part of the book visits also resource allocation.
Chapter 5 of Attiya and Welch’s book, and Chap. 9 of Kshemkalyani and
Singhal’s book, are fully devoted to the mutual exclusion problem in shared
Afterword 493

memory systems. Both books present also constructions of atomic read/write

registers from weaker registers.
– The construction of a shared memory on top of an asynchronous message-
passing system where processes may crash is presented in the previous
three books, and in Chap. 4 of a book by C. Cachin, R. Guerraoui, and
L. Rodrigues [60]. Such constructions constitute a main part of a book by
M. Raynal [236]. (More generally, this last book is entirely devoted to
failure-prone message-passing systems and the use of failure detectors to
circumvent impossibility results encountered in these systems.)
• Recent books entirely devoted to shared memory systems.
– Taubenfeld’s book [262] covers a lot of results motivated by problems which
arise when building systems. The book presents a lot of algorithms mainly
devoted to process coordination (mutual exclusion, resource allocation,
barrier synchronization, etc.). It also addresses timing-based synchronization
algorithms. An impressive number of results related to complexity bounds for
shared memory synchronization are also presented.
– Herlihy and Shavit’s book [146] is made up of two main parts. The first
addresses base principles of process synchronization and concurrent objects.
The second part is practice-oriented. It presents a huge number of algorithms
implementing concurrent objects which are encountered in a lot of
applications (e.g., linked lists, concurrent stacks, skip lists, priority queues).
All the algorithms are described in Java.

... Marco Polo describes a bridge, stone by stone.

‘But which is the stone that supports the bridge?’ Kublai Khan asks.
‘The bridge is not supported by one stone or another’, Marco answers,
‘but by the line of the arch that they form’.
In Invisible Cities (1963), Italo Calvino (1923–1985).

... Y colorin colorado, este cuento se ha acabado.

Anonymous, Spanish culture.