Consolidate executor_runners

[kirklandsign]

🚀 The feature, motivation and pitch

Now we have so many executor runner targets, one for each backend. We should use a main one, and the link backends.

If you look at https://github.com/search?q=repo%3Apytorch%2Fexecutorch+executor_runner+language%3ACMake&type=code&l=CMake you will see we have quite a few add_executable(xyz_executor_runner ...)

Often they contain the same source as the main executor runner:

executorch/CMakeLists.txt

Lines 765 to 775 in df8fc61

	if(EXECUTORCH_BUILD_EXECUTOR_RUNNER)
	# Baseline libraries that executor_runner will link against.
	set(_executor_runner_libs executorch gflags)
	if(EXECUTORCH_BUILD_KERNELS_OPTIMIZED)
	list(APPEND _executor_runner_libs optimized_native_cpu_ops_lib)
	elseif(EXECUTORCH_BUILD_CADENCE)
	list(APPEND _executor_runner_libs cadence_ops_lib)
	else()
	list(APPEND _executor_runner_libs portable_ops_lib)
	endif()

and the difference is they link to the different library.

In this code, we can see for XNNPACK, we can link to it

executorch/CMakeLists.txt

Lines 786 to 788 in df8fc61

	if(EXECUTORCH_BUILD_XNNPACK)
	list(APPEND _executor_runner_libs xnnpack_backend)
	endif()

Just do the similar stuff for all backends

• qualcomm
• arm
• vulkan
• openvino
• mediatek
• coreml
• mps

After that, can we remove the extra xyz_executor_runner?

• qualcomm_
• arm_
• vulkan_
• openvino_
• mediatek_
• coreml_
• mps_
• xnnpack_

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @larryliu0820 @jathu

[metascroy]

I think this is a good idea. I added CoreML support for the executor runner in March, and it looks like ScottW added XNNPACK support there as well in March: https://github.com/pytorch/executorch/blob/main/CMakeLists.txt#L799-L846

[cccclai]

What if different backends have different ways plan input/output memory?

[kirklandsign]

Is it possible that we make executor_runner generic?