Compiler creating debug.cfg, debug_custom.json, and esp32.svd files in my sketch folder under version 2.0.6

[HomeSpan]

Board

Any board

Device Description

Occurs on all Espressif development boards

Hardware Configuration

Nothing

Version

other

IDE Name

Arduino 1.8.19

Operating System

MacOs 13.1

Flash frequency

80 MHz

PSRAM enabled

no

Upload speed

921600

Description

In the latest release (2.0.6), whenever you compile a sketch, the compiler is dropping three new files into the sketch folder: debug.cfg, debug_custom.json, and esp32.svd. These files have never appeared when compiling with version 2.0.5 or earlier. Is there a way of turning the automatic output of these files?

Sketch

Any sketch.

Debug Message

None

Other Steps to Reproduce

Compile any sketch with Arduino IDE 1.8.19 using the latest Arduino-ESP32 version 2.0.6 and the compiler drops three new files, presumably used for debugging/testing, in to the sketch folder.

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[svdrummer]

I have just been asking this question on the Arduino forum. I have started getting these files being added.
debug.cfg
debug_custom.json
esp32.svd

[me-no-dev]

they are required for JTAG debugging with Arduino IDE 2.0.3+

[IeiuniumLux]

@me-no-dev, The above 3 files are also now generated with the classic Arduino 1.8.19 IDE during the uploading. Why does the former IDE now require those files too?

[me-no-dev]

@IeiuniumLux it does not, but we can not detect which version is the IDE in order to copy them only on the new one

[HomeSpan]

@me-no-dev, Can the output be an option that is selected from the tool menu, similar to setting the "Core Debug Level" or "Erase all Flash Before Sketch Upload"? Perhaps something like "Output JTAG files" where the default is "No" so that the files would never be output in version 1 of the Arduino IDE and would be optionally output in version 2?

Separately, is there anything in either "platform.txt" or "programmers.txt" that we can edit to disable the JTAG output?

[me-no-dev]

@HomeSpan I'm not sure if it's possible to be controlled by an option, but could be tried. You can remove those lines to disable copy to sketch folder

[tobozo]

does it have to be the sketch folder?

looks like an additional entry in .gitignore files will be required for the meantime

[me-no-dev]

@tobozo it has to be the sketch folder, because that is where the IDE is looking for debug_custom.json and is also the current folder for gdb/openocd (and where they can find the other files). Maybe this is a temporary thing and more flexibility will be added to the IDE in the future. I am looking into only doing it when debug is enabled also. I will add the files to gitignore in a bit.

[tobozo]

@me-no-dev I've edited the platform.txt as per your recommendation.

If suggesting a different place is possible, it would certainly make sense to find those debug files in the same folder as the ELF file (near the build.options.json), where some debug material already exists.

[me-no-dev]

@tobozo we have no control over that part or I would have used the build folder as well.
Anyway, I created this PR that adds gitignore and copies the files only if JTAG adapter is selected (or particular build variable is set to 1)

@IeiuniumLux @HomeSpan PTAL

[tobozo]

can those debug tools eventually be chained with Arduino 1.8.x or is it only functional with IDE 2.0.3+ ?

[me-no-dev]

@tobozo since they are using a particular extension that the new IDE includes, it can not be made to work with earlier versions. 2.0.x versions before 2.0.3 also have it, but it's older version and does not work correctly.

[tobozo]

@me-no-dev thanks for clarifying

what about using {runtime.ide.version} to add precision?

I don't know what's the {runtime.ide.version} value for IDE 2.0.3+ but for Arduino-1.8.19 it's 10819 so in my situation this works:

[[ {runtime.ide.version} -le 10819 || {build.copy_jtag_files} -eq 0 ]]

[per1234]

I don't know what's the {runtime.ide.version} value for IDE 2.0.3+

Unfortunately it is 10607: arduino/arduino-cli#725

[tobozo]

oh crap, so "fixing" it will break something else?

looks like it's directly conflicting with arduino-cli platform specs

{runtime.ide.version}: the version number of the Arduino IDE as a number (this 
uses two digits per version number component, and removes the points and 
leading zeroes, so Arduino IDE 1.8.3 becomes 01.08.03 which becomes 
runtime.ide.version=10803). When using Arduino development software other 
than the Arduino IDE, this is set to a meaningless version number.

(....)

Compatibility note: Versions before Arduino IDE 1.6.0 only used one digit per 
version number component in {runtime.ide.version} (so 1.5.9 was 159, not 10509).

[me-no-dev]

what about using {runtime.ide.version} to add precision?

Would have been nice to be able to filter menu items this way as well, but... cant :)
Anyway, current solution will disable it for all boards that do not have menu item enabling it and only Espressif's boards do have it (default is off). Should make it fine for everyone. I doubt we need more precision than that and the new IDE should be the one we support from now on anyway. It's much better and offers more options and features.

[dlarue]

How about only generating those files in the sketch folder when the menu item/option Sketch->Optimize for Debugging is checked/enabled?

I know it's not typical for sketches to be compiled for different board types but because the debugging setup files are being generated with every compile, specifically debug_custom.json, any debugging of that sketch with another board will error out unless it's another esp32 board.

[tobozo]

@dlarue that would sure save some github space

[edit] and user storage too, especially the ~/Arduino/libraries folder

[dlarue]

Wow @tobozo ya that'd be another side effect of only generating these JTAG debugger files, and project debugging symbols, etc no matter if the user is debugging or not. While I wish there were real menu options in the Tools menu for debugging, as the STM32 does, at the very least using the built-in Sketch->Optimize for Debugging seems a natural fit for this.

BTW, before Espressif devs released 2.0.6 which added the debugging capabilities, I was hacking the packages and board files to add the compiler options to the esp32 build and it was functional or at least I saw the -Os -O3 etc options showing up in compiler output. It seemed natural in the STM32 package to set debugging in the Tools menu and to have that kind of resolution of controlling the build process.

[dlarue]

For example, this is how the STM32 devs allow Arduino IDE developers to control the build system:

[me-no-dev]

We added a menu for selecting if JTAG is enabled or not. If not, it will not copy the files. It will come out soon in 2.0.7. Optimization flags are something else and should not mix them together.

[dlarue]

if just doing JTAG or !JTAG then hopefully you're leveraging what Arduino IDE devs put in place on the Sketch menu with the "Optimize for Debugging" flag.

Yes, compiler optimizations are another item and even I can do that by editing a couple of files and backing them up for when they get replaced by uprades like 2.0.6 -> 2.0.7.