Skip to content

Add AST section to examples #545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: ppx-by-example
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 52 additions & 13 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,38 +1,77 @@
# Define installation arguments with optional prefix
INSTALL_ARGS := $(if $(PREFIX),--prefix $(PREFIX),)
# Define examples with their descriptions
EXAMPLE_DESCRIPTIONS := \
"example-building-ast:Demonstrates how to build AST" \
"example-destructuring-ast:Demonstrates how to destructure an AST" \

# Default rule
default:
.PHONY: help
help: ## Print this help message
@echo "";
@echo "List of available make commands";
@echo "";
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf " \033[36m%-15s\033[0m %s\n", $$1, $$2}';
@echo "";
@echo "Available examples:";
@echo "";
@for desc in $(EXAMPLE_DESCRIPTIONS); do \
target=$$(echo $$desc | cut -d: -f1); \
description=$$(echo $$desc | cut -d: -f2); \
printf " \033[36m%-30s\033[0m %s\n" "$$target" "$$description"; \
done
@echo "";

.PHONY: default
default: ## Build the project with auto-promote
dune build --auto-promote @install

install:
.PHONY: install
install: ## Install the project
dune install $(INSTALL_ARGS)

uninstall:
.PHONY: uninstall
uninstall: ## Uninstall the project
dune uninstall $(INSTALL_ARGS)

reinstall: uninstall reinstall
.PHONY: reinstall
reinstall: ## Reinstall the project
uninstall reinstall

test:
.PHONY: test
test: ## Run tests
dune runtest

doc:
.PHONY: doc
doc: ## Build documentation
dune build @doc

clean:
.PHONY: doc-dev
doc-dev: ## Build and watch documentation
dune build @doc --watch & dune_pid=$$!; \
trap 'kill $$dune_pid' EXIT; \
sleep 2 && open _build/default/_doc/_html/index.html & \
wait $$dune_pid

.PHONY: clean
clean: ## Clean the build artifacts
dune clean

all-supported-ocaml-versions:
.PHONY: all-supported-ocaml-versions
all-supported-ocaml-versions: ## Build for all supported OCaml versions
dune build @install --workspace dune-workspace.dev --root .

opam-release:
.PHONY: opam-release
opam-release: ## Release the project using opam
dune-release distrib --skip-build --skip-lint --skip-tests
dune-release publish distrib --verbose
dune-release opam pkg
dune-release opam submit

bench:
.PHONY: bench
bench: ## Run benchmarks
dune build bench --profile release
dune exec bench/bench.exe

.PHONY: default install uninstall reinstall clean test doc bench
.PHONY: all-supported-ocaml-versions opam-release
.PHONY: $(TARGET)
example-%: ## Run example with specified target, e.g. make example-global-transformation
opam exec -- dune exec $*-example
173 changes: 173 additions & 0 deletions doc/example-ast-building.mld
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
{0 Building AST}

{1 Table of Contents}

- {{!section-description} Description}
- {{!section-"building-asts-with-pure-ocaml"} Building ASTs with Pure OCaml}
{ul {- {{!section-"example-building-a-simple-integer-ast-manually"} Example: Building a Simple Integer AST Manually}}}

- {{!section-"building-asts-with-ast_builder"} Building ASTs with {!Ppxlib.Ast_builder}}
{ul {- {{!section-"example-1-using-pexp_constant-for-integer-ast"} Example 1: Using [pexp_constant] for Integer AST}}}
{ul {- {{!section-"example-2-using-eint-for-simplified-integer-ast"} Example 2: Using [eint] for Simplified Integer AST}}}

- {{!section-"using-metaquot-for-ast-construction"} Using Metaquot for AST Construction}
{ul {- {{!section-"example-building-an-integer-ast-with-metaquot"} Example: Building an Integer AST with Metaquot}}}

- {{!section-"using-anti-quotations-in-metaquot"} Using Anti-Quotations in Metaquot}
{ul {- {{!section-"example-inserting-dynamic-expressions-with-anti-quotations"} Example: Inserting Dynamic Expressions with Anti-Quotations}}}

- {{!section-"building-complex-expressions"} Building Complex Expressions}
{ul {- {{!section-"example-1-constructing-a-let-expression-with-ast_builder"} Example 1: Constructing a Let Expression with [Ppxlib.Ast_builder]}}}
{ul {- {{!section-"example-2-constructing-a-let-expression-with-metaquot"} Example 2: Constructing a Let Expression with Metaquot}}}

- {{!section-conclusion} Conclusion}

{1:description Description}

Building an AST (Abstract Syntax Tree) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Building an AST (Abstract Syntax Tree) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.
Building an abstract syntax tree (AST) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.


For example, if you want to generate the following code:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me that it might be a little confusing to say "generate the following code", the reader may think this code is what the ppx might generate when in fact it seems to the be the starting point of the example. Maybe something like "For example, if we have the following code: ... and we wish to replace"


{[
let zero = [%int 0]
]}

and replace the extension point [%int 0] with [0] to produce [let zero = 0], you’ll need to build an AST that represents this transformation.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
and replace the extension point [%int 0] with [0] to produce [let zero = 0], you’ll need to build an AST that represents this transformation.
and replace the extension point [[%int 0]] with [0] to produce [let zero = 0], you’ll need to build an AST that represents this transformation.


There are several methods to build an AST. We’ll discuss three approaches:

- {b Building ASTs with Pure OCaml}
- {b Building ASTs with {!Ppxlib.Ast_builder}}
- {b Using Metaquot for AST Construction}

{1:building-asts-with-pure-ocaml Building ASTs with Low-Level Builders}

The most fundamental way to build an AST is to manually construct it using Low-Level Builders data structures.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a little confused what Low-Level Builders meant. Looking at the example it seems we could perhaps rephrase this as something like "building values from the {! Parsetree} directly" ?


{2:example-building-a-simple-integer-ast-manually Example: Building a Simple Integer AST Manually}

{{:https://github.com/ocaml-ppx/ppxlib/tree/main/examples/1-AST/a%20-%20Building%20AST/building_ast.ml#L5-L16} 🔗 Sample Code}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the latest version of mdx supports mld files, couldn't we use mdx to help keep the code snippets in sync with the examples?


{[
let zero ~loc : Ppxlib_ast.Ast.expression =
{
pexp_desc = Pexp_constant (Pconst_integer ("0", None));
pexp_loc = loc;
pexp_loc_stack = [];
pexp_attributes = [];
}
]}

While this method provides full control over the AST, it is verbose and less maintainable.

{1:building-asts-with-ast_builder Building ASTs with {!Ppxlib.Ast_builder}}

PPXLib provides the {!Ppxlib.Ast_builder} module, which simplifies the process of building ASTs by providing helper functions.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
PPXLib provides the {!Ppxlib.Ast_builder} module, which simplifies the process of building ASTs by providing helper functions.
The {!Ppxlib.Ast_builder} module simplifies the process of building ASTs by providing helper functions.


{2:example-1-using-pexp_constant-for-integer-ast Example 1: Using [pexp_constant] for Integer AST}

{{:https://github.com/ocaml-ppx/ppxlib/tree/main/examples/1-AST/a%20-%20Building%20AST/building_ast.ml#L18-L24} 🔗 Sample Code}

{[
let one ~loc =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whilst I like that each method of generating Parsetree nodes increments, it might be a little confusing given the context of this was given as replacing [%int 0] with 0?

Ast_builder.Default.pexp_constant ~loc (Parsetree.Pconst_integer ("1", None))
]}

This method is more readable and concise compared to the pure OCaml approach.

{2:example-2-using-eint-for-simplified-integer-ast Example 2: Using [eint] for Simplified Integer AST}

{{:https://github.com/ocaml-ppx/ppxlib/tree/main/examples/1-AST/a%20-%20Building%20AST/building_ast.ml#L26-L31} 🔗 Sample Code}

{[
let two ~loc = Ast_builder.Default.eint ~loc 2
]}

{b Tip:} [eint] is an abbreviation for expression ([e]) integer ([int]).

{1:using-metaquot-for-ast-construction Using Metaquot for AST Construction}

Metaquot is a syntax extension that allows you to write ASTs in a more natural and readable way.

{2:example-building-an-integer-ast-with-metaquot Example: Building an Integer AST with Metaquot}

{{:https://github.com/ocaml-ppx/ppxlib/tree/main/examples/1-AST/a%20-%20Building%20AST/building_ast.ml#L33-L38} 🔗 Sample Code}

{[
let three ~loc = [%expr 3]
]}

{b Tip:} Metaquot is highly readable and intuitive but is static. For dynamic values, use Anti-Quotations.

{2:using-anti-quotations-in-metaquot Using Anti-Quotations in Metaquot}

Anti-Quotations allow you to insert dynamic expressions into your Metaquot ASTs.

{3:example-inserting-dynamic-expressions-with-anti-quotations Example: Inserting Dynamic Expressions with Anti-Quotations}

{{:https://github.com/ocaml-ppx/ppxlib/tree/main/examples/1-AST/a%20-%20Building%20AST/building_ast.ml#L72-L77} 🔗 Sample Code}

{[
let anti_quotation_expr expr = [%expr 1 + [%e expr]]
]}

For example, to insert the AST for [1]:

{[
let _ =
print_endline
("\nLet expression with metaquot and anti-quotation: "
^ Astlib.Pprintast.string_of_expression (anti_quotation_expr (one ~loc)))
]}

{1:building-complex-expressions Building Complex Expressions}

Beyond simple expressions, you may need to build more complex ASTs, such as [let] expressions.

{2:example-1-constructing-a-let-expression-with-ast_builder Example 1: Constructing a Let Expression with {!Ppxlib.Ast_builder}}

{{:https://github.com/ocaml-ppx/ppxlib/tree/main/examples/1-AST/a%20-%20Building%20AST/building_ast.ml#L40-L60} 🔗 Sample Code}

{[
let let_expression =
let expression =
Ast_builder.Default.pexp_constant ~loc:Location.none
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can, I think we want to try and suggest best practices and not have write ~loc:Location.none, how about this becomes a let let_expression ~loc = or we open a module where ~loc is defined which is usually what we suggest doing?

(Pconst_integer ("3", None))
in
let pattern =
Ast_builder.Default.ppat_var ~loc:Location.none
(Ast_builder.Default.Located.mk ~loc:Location.none "foo")
in
let let_binding =
Ast_builder.Default.value_binding ~loc:Location.none ~pat:pattern
~expr:expression
in
Ast_builder.Default.pexp_let ~loc:Location.none Nonrecursive [ let_binding ]
(Ast_builder.Default.eunit ~loc:Location.none)
]}

{2:example-2-constructing-a-let-expression-with-metaquot Example 2: Constructing a Let Expression with Metaquot}

{{:https://github.com/ocaml-ppx/ppxlib/tree/main/examples/1-AST/a%20-%20Building%20AST/building_ast.ml#L62-L70} 🔗 Sample Code}

{[
let let_expression =
[%expr
let foo = 3 in
()]
]}

This approach is shorter and easier to understand.

{1:conclusion Conclusion}

In this section, we explored three methods for building ASTs:

- {b Pure OCaml}: The most basic but verbose approach.
- {b Using {!Ppxlib.Ast_builder}}: A more readable and maintainable option.
- {b Using Metaquot}: The most intuitive method, especially when combined with Anti-Quotations for dynamic values.

Each method has its strengths, so choose the one that best fits your needs. Understanding all three will give you greater flexibility in creating effective and maintainable PPXs.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should tell our users to use Ast_builder or Metaquot -- having just gone through a big merge for the internal AST bump we have greater control and flexibility in helping users migrate if they use these two approaches over building AST nodes. What do you think @pedrobslisboa ?


{2 Next Steps}
On the next section, we will learn how to destructure an AST. {{!page-"example-ast-destructing"} Read more}
Loading