Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions website/contribute/developer-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ IntelliJ is the recommended IDE for developing Hudi. To contribute, you would ne
![IDE setup maven 1](/assets/images/contributing/IDE_setup_maven_1.png)
![IDE setup maven 2](/assets/images/contributing/IDE_setup_maven_2.png)

- If you switch maven build profile, e.g., from Spark 3.2 to Spark 3.3, you need to first build Hudi in the command line first and `Reload All Maven Projects` in IntelliJ like below,
- If you switch maven build profile, e.g., from Spark 3.4 to Spark 3.5, you need to first build Hudi in the command line first and `Reload All Maven Projects` in IntelliJ like below,
so that IntelliJ re-indexes the code.

![IDE setup reload](/assets/images/contributing/IDE_setup_reload.png)
Expand All @@ -48,7 +48,7 @@ so that IntelliJ re-indexes the code.
* Install and activate CheckStyle plugin

![IDE_setup_checkstyle_1](/assets/images/contributing/IDE_setup_checkstyle_1.png)
* In `Settings` > `Tools` > `Checkstyle`, use a recent version, e.g., 10.17.0
* In `Settings` > `Tools` > `Checkstyle`, use a recent version, e.g., 12.1.0

![IDE_setup_checkstyle_2](/assets/images/contributing/IDE_setup_checkstyle_2.png)
* Click on `+`, add the style/checkstyle.xml file, and name the configuration as "Hudi Checks"
Expand All @@ -70,6 +70,9 @@ so that IntelliJ re-indexes the code.
- \[Recommended\] As it is required to add [Apache License header](https://www.apache.org/legal/src-headers#headers) to all source files, configuring "Copyright" settings as shown below will come in handy.

![IDE setup copyright 1](/assets/images/contributing/IDE_setup_copyright_1.png)



![IDE setup copyright 2](/assets/images/contributing/IDE_setup_copyright_2.png)


Expand All @@ -81,7 +84,7 @@ Listing out some of the maven commands that could be useful for developers.
```shell
mvn clean package -DskipTests
```
Default profile is spark2 and scala2.11
Default profile is spark3.5 and scala2.12

- For continuous development, you may want to build only the modules of interest. for eg, if you have been working with
Hudi Streamer, you can build using this command instead of entire project. Majority of time goes into building all different bundles we have
Expand Down Expand Up @@ -162,11 +165,11 @@ This Quick Video will give a code walkthrough to start with [watch](https://www.
When submitting a PR please make sure to NOT commit the changes mentioned in these steps, instead once testing is done make sure to revert the changes and then submit a pr.
:::

0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.2 use `mvn clean package -Dspark3.2 -Dscala-2.12 -DskipTests`.
0. Build the project with the intended profiles via the `mvn` cli, for example for spark 3.5 use `mvn clean package -Dspark3.5 -Dscala-2.12 -DskipTests`.
1. Install the "Maven Helper" plugin from the Intellij IDE.
2. Make sure IDEA uses Maven to build/run tests:
* You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark2.4` and `scala-2.11` or `spark3.2`, `scala-2.12` etc.
* Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.2` `-Dscala-2.12`
* You need to select the intended Maven profiles (using Maven tool pane in IDEA): select profiles you are targeting for example `spark3.4` and `scala-2.2` or `spark3.5`, `scala-2.12` etc.
* Add `.mvn/maven.config` file at the root of the repo w/ the the profiles you selected in the pane: `-Dspark3.5` `-Dscala-2.12`
* Add `.mvn/` to the `.gitignore` file located in the root of the project.
3. Make sure you change (temporarily) the `scala.binary.version` in the root `pom.xml` to the intended scala profile version. For example if running with spark3 `scala.binary.version` should be `2.12`
4. Finally right click on the unit test's method signature you are trying to run, there should be an option with a mvn symbol that allows you to `run <test-name>`, as well as an option to `debug <test-name>`.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/static/assets/images/contributing/IDE_setup_java.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/static/assets/images/contributing/IDE_setup_maven_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/static/assets/images/contributing/IDE_setup_reload.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.