Python Mini Reference: A Hitchhiker's Guide to the Modern Programming Languages, #3

Preface

Python is a dynamic language.

It means many different things. It means that the language is more flexible. It means that Python is an easier language to program with. You can quickly write a simple program without having to go through too much rituals, compared to many other programming languages. Python scripts tend to require less boilerplate code. Python gives you more freedom.

On the other hand, it also means that the language is less safe, and more error prone. It means that it is harder to build a larger system with Python. It means that software written in Python is generally harder to maintain over a longer period of time.

The key to using Python effectively is to understand this tradeoff. Python can be an ideal language, for instance, for quick prototyping, or scripting. On the flip side, Python is not as much used for the enterprise software development.

Python is a general purpose programming language. Python is used in many different application areas, from system administration tasks to web application development. Python is now one of the most widely used programming languages for scientists, who have traditionally been using more high-level tools like Matlab. Python provides an easier upgrade path to these non-professional programmers. Now, Python is becoming the language for machine learning and data science.

Python is beginner-friendly. In fact, it seems to be the most favorite first language for beginning programmers, even more so than JavaScript.

It should be noted, however, that Python is not a simple language. Over the past 30 years or so of its history, it has gone through a lot of big and small changes. It is still easy to get started with programming in Python. Nonetheless, once you reach a certain point, say, from the advanced beginners to intermediate level, its complexity can be overwhelming.

This book will give you a broad and sanitized overview of the Python language, as of its most recent incarnation (e.g., 3.10 and 3.11). This book can be useful to anyone who has been dabbling with Python without solid foundation. It can also be useful to the people who have experience with other programming languages and want to get some quick overview of the language.

This book is written as an (informal) language reference. The goal of this book is not to teach you how to program effectively in Python, but rather to provide a concise summary of the language grammar. If you have some basic programming knowledge, you can read this book more or less from beginning to end, and you should be able to get the overall understanding of the Python programming language quickly, regardless of your particular programming background.

The book is written for a broad audience, but one caveat is that there are a fair amount of cross references, unlike in the books written in a tutorial style. If you have no prior experience with programming in Python or any similar language, then you may find it a little difficult to go through some parts of the book. This book is not for complete beginners. It skims through some elementary concepts in the beginning, for the sake of brevity, so that we can focus more on the intermediate and advanced level topics.

This book is not an authoritative language reference. For that, we recommend the readers to refer to the official language specification.

1. Introduction

This book will give you an overview of the Python language grammar.

It appears that the line between the programming language proper and the standard library is becoming more blurred these days. Although it is not our goal to go through the Python standard library (which is well beyond the scope of this book), we will touch upon a few important concepts from the standard library modules that are considered more or less part of the language.

This reference starts with the most boring part of Python. If you plan to read the book from beginning to end, say, rather than using it just as a quick reference, then you can skip the first few chapters in your first reading, and come back to them later when needed.

The term program means different things in different contexts, and across different programming languages. We start with somewhat formal explanations of what a Python program is, and how Python programs are executed, e.g., in the Python interpreter.

Python programs are logically organized into packages and modules, which we take a look at next. A module corresponds to a file in a physical file system, and modules, and including package modules, are the basic units of code reuse and sharing in Python.

Next, we briefly go through some of the lexical elements of the Python source code. There are some small variations across different programming languages, but their lexical compositions are rather similar, and Python is no different. This part can be skipped in your reading.

Generally speaking, a program consists of code and data. Code refers to instructions. Data in Python is represented by objects. The object in Python is a fundamental component. Everything which we deal with in a Python program is objects. We start the main part of the book by introducing various important concepts related to the objects.

Although Python uses a dynamic type system, the types still play the foundational roles in the Python programming language. We first go through some of the basic builtin types in Python. Python includes quite a few builtin types, and this reference touches on only some of them. As indicated, this is generally true across all topics discussed in this reference. Completeness is not the goal of this mini reference.

Python also includes a few builtin compound types such as list and dictionary, which are important components of any non-trivial programs. We briefly go through these types in the next chapter. Advanced types, e.g., functions and classes, in particular, are explained in detail later in the book.

As with any imperative programming language, Python has expressions and statements. An expression prescribes how to compute a value using other expressions and values. Python supports most of the common operators and expressions found in other imperative languages. If you are coming from other procedural programming language background, you can skim through most of this part.

A statement is an instruction. Statements control the flow of a program to attain the desired goal. In Python, there are two kinds of statements, simple statements and compound statements. Compound, or complex, statements can include other simple or compound statements.

Simple statements include expression statement, assignment statement, assert statement, pass statement, del statement, return statement, raise statement, break statement, continue statement, global statement, and nonlocal statement. Python’s compound statements include if statement, while statement, for statement, try statement, with statement, match statement, function def statement, class definition, and other coroutine-related statements.

One of the most significant changes to Python for the last 30+ years has been the addition of structural pattern matching to the language, as of Python 3.10 (2021). Pattern matching was first popularized by functional programming languages like Haskell, and it is now becoming more and more widely available across many different programming languages like C#, rust, swift, scala, and (yes) even Java.

As we more adopt this feature, as a community, pattern matching will likely change how we program in Python in the coming years. Although it is currently supported only in the context of the match statement, it is conceivable, and in fact expected, that pattern matching will be available more broadly across the language in the near future, considering its simplicity, elegance, and power.

A function definition is a compound statement. We dedicate its own chapter to the function definition. This chapter also includes other function-related topics such as lambda expressions and decorators.

Another compound statement, the class definition, is explained next. Other class-related concepts such enums and data classes are described in this chapter as well. We also provide an informal introduction to the object oriented programming styles in Python, including multiple inheritance.

Special kinds of functions, generators and coroutines, are separately discussed in the last part of the book, in the Coroutines chapter. We briefly touch on the (high-level) asynchronous programming style in Python using the new async and await keywords. In the last section, we provide a simple async programming example, namely, the producer-consumer problem.

As stated, we do not exhaustively cover every topic in Python in this mini reference, including the (low-level) multi-thread and multi-process programming, etc. However, it goes through all the essential language features that any intermediate to advanced level Python programmers should be familiar with.

2. Python Programs

A complete Python program can be passed to the Python interpreter:

With the -c command line option with a program text,

With the -m command line option with a module name,

As a file name passed as the first command line argument, or

From the standard input.

The Python interpreter executes the input file, or the input text, as a complete program, in the namespace of a special module, __main__.

When the file or standard input is a terminal, or when the Python interpreter is invoked without an argument, the interpreter enters the interactive mode. The Python interpreter reads and executes one statement at a time (simple or compound) in the interactive mode instead of executing a complete program.

Here are a few examples:

①

A complete Python program is executed using the -c flag.

①

The Python script is passed to the interpreter as a command line argument.

①

The Python module hello_world is executed through the -m flag.

①

The Python code from the stdin (via Unix pipe).

①

Using the Unix shell here string syntax.

①

The here doc.

①

Invoking the python command without an argument starts the REPL.

①

Ditto. The Python interpreter ignores the flags following the dash -.

①

It also starts the