Skip to content

Add torchao kernels to xcframework #10694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
metascroy opened this issue May 5, 2025 · 0 comments
Open

Add torchao kernels to xcframework #10694

metascroy opened this issue May 5, 2025 · 0 comments
Labels
good first issue Good for newcomers module: build/install Issues related to the cmake and buck2 builds, and to installing ExecuTorch module: ios Issues related to iOS code, build, and execution module: quantization Issues related to quantization triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@metascroy
Copy link
Contributor

metascroy commented May 5, 2025

🚀 The feature, motivation and pitch

We can build low-bit kernels on Macbook (https://github.com/pytorch/executorch/tree/main/examples/models/llama#running-with-low-bit-kernels), but we don't currently build these as part of the main ExecuTorch CMakeLists.txt.

Instead they are built inside the runner code here: https://github.com/pytorch/executorch/blob/main/examples/models/llama/CMakeLists.txt#L118-L137

In this task, we want to add an option to build this kernel library to the ET CMakeLists.txt (https://github.com/pytorch/executorch/blob/main/CMakeLists.txt) guarded under EXECUTORCH_BUILD_KERNELS_TORCHAO, and add them to the xcframeworks (https://github.com/pytorch/executorch/blob/main/scripts/build_apple_frameworks.sh).

This will let us run low-bit kernels on the iOS app.

cc @larryliu0820 @jathu @kimishpatel @jerryzh168 @shoumikhin @cbilgin

@metascroy metascroy added the good first issue Good for newcomers label May 5, 2025
@metascroy metascroy changed the title Add torchao kernels Add torchao kernels to xcframework May 5, 2025
@Jack-Khuu Jack-Khuu added module: quantization Issues related to quantization triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels May 6, 2025
@metascroy metascroy added module: build/install Issues related to the cmake and buck2 builds, and to installing ExecuTorch module: ios Issues related to iOS code, build, and execution labels May 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers module: build/install Issues related to the cmake and buck2 builds, and to installing ExecuTorch module: ios Issues related to iOS code, build, and execution module: quantization Issues related to quantization triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
Development

No branches or pull requests

2 participants