Add AST section to examples #545

pedrobslisboa · 2024-12-15T21:59:48Z

41 files where: 20 are pngs and 8 are old examples files deleted. So don't be scared

Description

This is the first part of adding the ppx-by-example to ppxlib
I will divide it into PR so we can review it better, focusing on the explanation and typos without having tons of files to review. It is really necessary that this section is easy to understand and is aligned with ppxlib.
@NathanReb Can you create a ppx-by-example base branch? So I can point this one to it, and after all merges (Other sections) on this branch, we move with the entire ppx-by-example branch to main?

AST

This PR adds the AST sections AST / Building ASTs / Destructing ASTs

How to test it (It's important to check it not only by code but also as styles)

At this branch run

 make doc
 open _build/default/_doc/_html/ppxlib/index.html

Access Examples
Access each section:
- AST
- Building an AST
- Destructing an AST

pedrobslisboa · 2024-12-23T15:28:22Z

Makefile

@NathanReb I've made some improvements on Makefile, can you check out if it makes sense?

The main feature is to add doc-dev, but I also added the help command

pedrobslisboa · 2024-12-23T15:29:18Z

@patricoferris Tagging you for the review as you reacted to this PR

pedrobslisboa · 2024-12-23T15:44:23Z

doc/dune

+     (system "mkdir -p %{project_root}/_doc/_html/ppxlib/assets/images")
+     (system "cp -R ./images/ %{project_root}/_doc/_html/ppxlib/assets/images/"))))


Btw, I didn't know any other way to get assets on odoc. Do you guys happen to know?

@NathanReb @patricoferris I know that Odoc v3 supports it; maybe there is no alternative other than due to a hack similar to this one. Do you guys know a better hack?

Indeed, this hack will unfortunately not work for the documentation generated for ocaml.org.

The proper way to do it is to:

install the image files in the odoc-pages folder of the doc (see this example for how to do it with dune)

Include it with the odoc3 syntax, in your case {image!filename.png}

The tooling support for for odoc 3 is still under construction, so dune won't yet be able to build the documentation with the images... however, odoc_driver can and soon dune and ocaml.org will be able to do it! You would be one of the first users of images the odoc 3 way!

(Another way to do it is to use permalinks to the image eg {image:https://github.com/ocaml-ppx/ppxlib/<hash>/filename.png})

I also wonder if turning some images into text wouldn't be better for accessibility.

@panglesd Thanks for the review. I followed the idea of having it as text/code instead, and I loved how it is right now.

Before After

The only sad part is that the Highlight is missing. However, it is still cleaner, and people can copy/paste the code.

NathanReb · 2025-01-06T15:00:46Z

@pedrobslisboa just pushed a ppx-by-example branch, sorry for the delay!

Signed-off-by: Pedro B S Lisboa <[email protected]>

NathanReb · 2025-05-27T08:42:53Z

I just updated the ppx-by-example branch and rebased it on top of main.

Can you please rebase this @pedrobslisboa ? I'll review straight away!

Signed-off-by: Pedro B S Lisboa <[email protected]>

pedrobslisboa · 2025-05-28T19:36:49Z

Makefile

@NathanReb, I've made some improvements on Makefile. Does it make sense?

The main feature is to add the example to the makefile, but I also added the help command.

pedrobslisboa · 2025-05-28T19:37:44Z

I just updated the ppx-by-example branch and rebased it on top of main.

Can you please rebase this @pedrobslisboa ? I'll review straight away!

Rebased and ready for review.

Signed-off-by: Pedro B S Lisboa <[email protected]>

pedrobslisboa · 2025-06-06T14:24:49Z

🏓

patricoferris

Thanks for all of the hardwork here -- I've started a review. In general, I like the idea of having more documentation like this, but I do worry about the maintenance overhead if we don't try to automate some of this.

patricoferris · 2025-02-18T10:52:42Z

doc/example-ast-building.mld

+
+{1:building-asts-with-pure-ocaml Building ASTs with Low-Level Builders}
+
+The most fundamental way to build an AST is to manually construct it using Low-Level Builders data structures.


I was a little confused what Low-Level Builders meant. Looking at the example it seems we could perhaps rephrase this as something like "building values from the {! Parsetree} directly" ?

patricoferris · 2025-02-18T10:55:12Z

doc/example-ast-building.mld

+- {b Using {!Ppxlib.Ast_builder}}: A more readable and maintainable option.
+- {b Using Metaquot}: The most intuitive method, especially when combined with Anti-Quotations for dynamic values.
+
+Each method has its strengths, so choose the one that best fits your needs. Understanding all three will give you greater flexibility in creating effective and maintainable PPXs.


I think we should tell our users to use Ast_builder or Metaquot -- having just gone through a big merge for the internal AST bump we have greater control and flexibility in helping users migrate if they use these two approaches over building AST nodes. What do you think @pedrobslisboa ?

patricoferris · 2025-06-06T22:55:48Z

doc/example-ast-building.mld

+
+{1:description Description}
+
+Building an AST (Abstract Syntax Tree) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.


Suggested change

Building an AST (Abstract Syntax Tree) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.

Building an abstract syntax tree (AST) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.

patricoferris · 2025-06-06T22:58:17Z

doc/example-ast-building.mld

+
+Building an AST (Abstract Syntax Tree) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.
+
+For example, if you want to generate the following code:


It seems to me that it might be a little confusing to say "generate the following code", the reader may think this code is what the ppx might generate when in fact it seems to the be the starting point of the example. Maybe something like "For example, if we have the following code: ... and we wish to replace"

patricoferris · 2025-06-06T22:58:40Z

doc/example-ast-building.mld

+let zero = [%int 0]
+]}
+
+and replace the extension point [%int 0] with [0] to produce [let zero = 0], you’ll need to build an AST that represents this transformation.


Suggested change

and replace the extension point [%int 0] with [0] to produce [let zero = 0], you’ll need to build an AST that represents this transformation.

and replace the extension point [[%int 0]] with [0] to produce [let zero = 0], you’ll need to build an AST that represents this transformation.

patricoferris · 2025-06-10T08:58:01Z

examples/1-AST/a - Building AST/building_ast.ml

@@ -0,0 +1,80 @@
+open Ppxlib


I wonder if it would make more sense if these code snippets lived alongside the docs themselves. In conjunction with mdx that could help keep everything up to date?

The previous examples that lived here were more end-to-end examples that I think should be kept.

patricoferris · 2025-06-10T09:03:22Z

doc/example-ast-building.mld

+{[
+let let_expression =
+  let expression =
+    Ast_builder.Default.pexp_constant ~loc:Location.none


If we can, I think we want to try and suggest best practices and not have write ~loc:Location.none, how about this becomes a let let_expression ~loc = or we open a module where ~loc is defined which is usually what we suggest doing?

patricoferris · 2025-06-10T09:07:29Z

doc/example-ast-destructing.mld

+
+{1:ast-structure-pattern-matching AST Structure Pattern Matching}
+
+The most fundamental method for destructuring an AST in PPXLib is by directly matching on the AST’s structure.


Looking through the current documentation and README, it seems to me that we tend to write Ppxlib rather than PPXLib. Perhaps we could even change all occurrences of PPXLib into {! Ppxlib}.

patricoferris · 2025-06-10T09:13:20Z

doc/example-ast-destructing.mld

+  pstr (pstr_eval (eint (int 1)) nil ^:: nil)
+]}
+
+Using [eint] instead of [pexp_constant] and [pconst_integer] provides better type safety. The [int] wildcard captures the integer value.


Suggested change

Using [eint] instead of [pexp_constant] and [pconst_integer] provides better type safety. The [int] wildcard captures the integer value.

Using [eint] instead of [pexp_constant] and [pconst_integer] provides better type safety. The [int] combinator captures the integer value.

I'm not entirely sure about the claim of "better type safety", I think eint is just a short-hand for the same code:

ppxlib/src/ast_pattern.ml

Line 226 in b3c240e

let eint t = pexp_constant (const_int t)

?

patricoferris · 2025-06-10T09:13:47Z

doc/example-ast-destructing.mld

+You can further simplify it:
+
+{[
+let match_int_payload =


Can we further simplify this to:

let match_int_payload = let open Ast_pattern in single_expr_payload (eint (int 1))

pedrobslisboa force-pushed the examples/ast branch from 3c7b85f to e5e4945 Compare December 15, 2024 22:07

pedrobslisboa marked this pull request as ready for review December 15, 2024 22:07

pedrobslisboa requested review from ceastlund, NathanReb, panglesd, patricoferris and pitag-ha as code owners December 15, 2024 22:07

pedrobslisboa mentioned this pull request Dec 15, 2024

[WIP] Add ppx-by-example #544

Closed

2 tasks

pedrobslisboa force-pushed the examples/ast branch 5 times, most recently from 6a2f1e9 to 6ad21fe Compare December 16, 2024 14:03

pedrobslisboa force-pushed the examples/ast branch from 6ad21fe to 160f335 Compare December 23, 2024 15:24

pedrobslisboa commented Dec 23, 2024

View reviewed changes

pedrobslisboa force-pushed the examples/ast branch 5 times, most recently from 7bf1887 to 2aeef20 Compare December 27, 2024 21:12

pedrobslisboa changed the base branch from main to ppx-by-example January 6, 2025 21:33

pedrobslisboa force-pushed the examples/ast branch from 2aeef20 to e585e65 Compare January 21, 2025 06:12

Start ppx-by-example doc

77aa068

Signed-off-by: Pedro B S Lisboa <[email protected]>

pedrobslisboa force-pushed the examples/ast branch 2 times, most recently from 737d517 to f08cc55 Compare May 2, 2025 12:04

pedrobslisboa force-pushed the examples/ast branch from f08cc55 to 8ce22a4 Compare May 28, 2025 19:12

pedrobslisboa added 2 commits May 28, 2025 16:32

Add ast sample

df18649

Signed-off-by: Pedro B S Lisboa <[email protected]>

Delete previous examples

d304a96

Signed-off-by: Pedro B S Lisboa <[email protected]>

pedrobslisboa force-pushed the examples/ast branch from 8ce22a4 to 91b7524 Compare May 28, 2025 19:32

pedrobslisboa commented May 28, 2025

View reviewed changes

pedrobslisboa force-pushed the examples/ast branch from 91b7524 to fc8fb70 Compare May 29, 2025 18:06

Improve Makefile

c215825

Signed-off-by: Pedro B S Lisboa <[email protected]>

pedrobslisboa force-pushed the examples/ast branch from fc8fb70 to c215825 Compare May 29, 2025 18:28

patricoferris reviewed Jun 10, 2025

View reviewed changes

		(system "mkdir -p %{project_root}/_doc/_html/ppxlib/assets/images")
		(system "cp -R ./images/ %{project_root}/_doc/_html/ppxlib/assets/images/"))))


		{1:building-asts-with-pure-ocaml Building ASTs with Low-Level Builders}

		The most fundamental way to build an AST is to manually construct it using Low-Level Builders data structures.


		{1:description Description}

		Building an AST (Abstract Syntax Tree) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.

	Building an AST (Abstract Syntax Tree) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.
	Building an abstract syntax tree (AST) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.


		Building an AST (Abstract Syntax Tree) is a fundamental part of creating a PPX in OCaml. You'll need to construct an AST to represent the code you want to generate or transform.

		For example, if you want to generate the following code:

	and replace the extension point [%int 0] with [0] to produce [let zero = 0], you’ll need to build an AST that represents this transformation.
	and replace the extension point [[%int 0]] with [0] to produce [let zero = 0], you’ll need to build an AST that represents this transformation.


		{1:ast-structure-pattern-matching AST Structure Pattern Matching}

		The most fundamental method for destructuring an AST in PPXLib is by directly matching on the AST’s structure.

	Using [eint] instead of [pexp_constant] and [pconst_integer] provides better type safety. The [int] wildcard captures the integer value.
	Using [eint] instead of [pexp_constant] and [pconst_integer] provides better type safety. The [int] combinator captures the integer value.

Add AST section to examples #545

Are you sure you want to change the base?

Add AST section to examples #545

Uh oh!

Conversation

pedrobslisboa commented Dec 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

41 files where: 20 are pngs and 8 are old examples files deleted. So don't be scared

Description

AST

How to test it (It's important to check it not only by code but also as styles)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pedrobslisboa commented Dec 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pedrobslisboa May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NathanReb commented Jan 6, 2025

Uh oh!

NathanReb commented May 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pedrobslisboa commented May 28, 2025

Uh oh!

pedrobslisboa commented Jun 6, 2025

Uh oh!

patricoferris left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pedrobslisboa commented Dec 15, 2024 •

edited

Loading

pedrobslisboa commented Dec 23, 2024 •

edited

Loading

pedrobslisboa May 28, 2025 •

edited

Loading