Skip to content

Arm build changes #5789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
May 18, 2021
Merged

Arm build changes #5789

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
07acc68
arm testing
michaelgsharp Mar 3, 2021
b813fb9
initial commit with build working on arm64
michaelgsharp Apr 5, 2021
c945a05
windows changes
michaelgsharp May 4, 2021
37ca2d7
build fixes for arm/arm64 with cross compilation
michaelgsharp May 6, 2021
745088b
cross build instructions added
michaelgsharp May 7, 2021
a50ba08
renamed arm to Arm. Changed TargetArchitecture to default to OS archi…
michaelgsharp May 7, 2021
f277fbf
fixed some formatting
michaelgsharp May 7, 2021
266208a
fixed capitilization
michaelgsharp May 7, 2021
3e27e67
fixed Arm Capitilization
michaelgsharp May 7, 2021
0b461b2
Fix cross-compilation if statement
michaelgsharp May 8, 2021
955c40b
building on apple silicon
michaelgsharp May 8, 2021
630e024
removed non build related files
michaelgsharp May 10, 2021
07e1fe0
Changes from PR comments. Removal of FastTreeNative flag.
michaelgsharp May 13, 2021
73c3f0f
Changes from pr comments.
michaelgsharp May 17, 2021
629b4e6
Fixes from PR comments.
michaelgsharp May 17, 2021
b207b56
Changed how we are excluding files.
michaelgsharp May 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
build fixes for arm/arm64 with cross compilation
  • Loading branch information
@michaelgsharp
michaelgsharp committed May 17, 2021
commit 37ca2d7ead3535aae05b0983ee2595d458bef5f4
758 changes: 0 additions & 758 deletions Microsoft.ML.sln
View file
Open in desktop

Large diffs are not rendered by default.

24 changes: 22 additions & 2 deletions docs/building/windows-instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ You can build ML.NET either via the command line or by using Visual Studio.
2. **[CMake](https://cmake.org/)** must be installed from [the CMake download page](https://cmake.org/download/#latest) and added to your path.

### Visual Studio 2019 Installation
We have successfully verified the below build instructions for Visual Studio version 16.4 and higher.
We have successfully verified the below build instructions for Visual Studio version 16.4 and higher.

#### Visual Studio 2019 - 'Workloads' based install

Expand Down Expand Up @@ -37,6 +37,12 @@ The following are the minimum requirements:
* .NET Framework 4.6 Targeting Pack
* Windows Universal CRT SDK

#### Visual Studio 2019 - Cross compilation for ARM

If you want to cross compile for arm you will also need from the 'Individual components' section:
* MSVC v142 - VS 2019 C++ ARM build tools
* MSVC v142 - VS 2019 C++ ARM64 build tools

## Building Instructions

In order to fetch dependencies which come through Git submodules the following command needs to be run before building: `git submodule update --init`.
Expand All @@ -51,7 +57,7 @@ After successfully running the command, the project can be built directly from t

### Building From the Command Line

You can use the Developer Command Prompt, Powershell or work in any regular cmd. The Developer Command Prompt will have a name like "Developer Command Prompt for VS 2019" or similar in your start menu.
You can use the Developer Command Prompt, Powershell or work in any regular cmd. The Developer Command Prompt will have a name like "Developer Command Prompt for VS 2019" or similar in your start menu.

From a (non-admin) Command Prompt window:

Expand All @@ -61,6 +67,20 @@ From a (non-admin) Command Prompt window:

**Note**: Before working on individual projects or test projects you **must** run `build.cmd` from the root once before beginning that work. It is also a good idea to run `build.cmd` whenever you pull a large set of unknown changes into your branch.

### Cross compiling for ARM From the Command Line

You can use the Developer Command Prompt, Powershell or work in any regular cmd. The Developer Command Prompt will have a name like "Developer Command Prompt for VS 2019" or similar in your start menu.

From a (non-admin) Command Prompt window based on what you want to target:

- `build.cmd /p:TargetArchitecture=arm`
- `build.cmd /p:TargetArchitecture=arm64`

You can then pack them into nugets, pick the same target architecture you built with:

- `build.cmd /p:TargetArchitecture=arm -pack`
- `build.cmd /p:TargetArchitecture=arm64 -pack`

## Running Tests

### Running tests from Visual Studio
Expand Down
8 changes: 4 additions & 4 deletions docs/samples/Microsoft.ML.Samples.GPU/Microsoft.ML.Samples.GPU.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@
<ProjectReference Include="..\..\..\src\Microsoft.ML.StandardTrainers\Microsoft.ML.StandardTrainers.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.ML.Transforms\Microsoft.ML.Transforms.csproj" />

<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="FastTreeNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="FastTreeNative" />
<NativeAssemblyReference Include="MatrixFactorizationNative" />
<NativeAssemblyReference Include="LdaNative" />
<NativeAssemblyReference Include="SymSgdNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="MklProxyNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="SymSgdNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="MklProxyNative" />

</ItemGroup>

Expand Down
8 changes: 4 additions & 4 deletions docs/samples/Microsoft.ML.Samples/Microsoft.ML.Samples.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,12 @@
<ProjectReference Include="..\..\..\src\Microsoft.ML.Core\Microsoft.ML.Core.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.ML.Data\Microsoft.ML.Data.csproj" />

<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="FastTreeNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="FastTreeNative" />
<NativeAssemblyReference Include="MatrixFactorizationNative" />
<NativeAssemblyReference Include="LdaNative" />
<NativeAssemblyReference Include="SymSgdNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="MklProxyNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="SymSgdNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="MklProxyNative" />
<None Include="Dynamic\Trainers\BinaryClassification\FastForest.cs">
<DesignTime>True</DesignTime>
<AutoGen>True</AutoGen>
Expand Down
4 changes: 4 additions & 0 deletions eng/common/cross/build-rootfs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,10 @@ __UbuntuPackages+=" libssl-dev"
__UbuntuPackages+=" zlib1g-dev"
__UbuntuPackages+=" libldap2-dev"

# ML.NET dependencies
__UbuntuPackages+=" libomp5"
__UbuntuPackages+=" libomp-dev"

__AlpinePackages+=" curl-dev"
__AlpinePackages+=" krb5-dev"
__AlpinePackages+=" openssl-dev"
Expand Down
10 changes: 5 additions & 5 deletions src/Microsoft.ML.Console/Microsoft.ML.Console.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,14 @@
<ProjectReference Include="..\Microsoft.ML.TensorFlow\Microsoft.ML.TensorFlow.csproj" />
<ProjectReference Include="..\Microsoft.ML.Transforms\Microsoft.ML.Transforms.csproj" />

<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="FastTreeNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="FastTreeNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="CpuMathNative" />
<NativeAssemblyReference Include="MatrixFactorizationNative" />
<NativeAssemblyReference Include="LdaNative" />
<!-- TODO: once we fix the 4 intel MKL methods, SymSgdNative will need to go back in. -->
<NativeAssemblyReference Include="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'SymSgdNative"/>
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="MklImports"/>
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm' And '$(OS)' == 'Windows_NT'" Include="libiomp5md"/>
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="SymSgdNative"/>
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="MklImports"/>
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm' And '$(OS)' == 'Windows_NT'" Include="libiomp5md"/>
</ItemGroup>

</Project>
2 changes: 1 addition & 1 deletion src/Microsoft.ML.FastTree/Microsoft.ML.FastTree.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<IncludeInPackage>Microsoft.ML.FastTree</IncludeInPackage>
<PackageDescription>ML.NET component for FastTree</PackageDescription>
<DefineConstants>$(DefineConstants);NO_STORE;CORECLR</DefineConstants>
<DefineConstants Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'">$(DefineConstants);USE_FASTTREENATIVE</DefineConstants>
<DefineConstants Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'">$(DefineConstants);USE_FASTTREENATIVE</DefineConstants>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is going to work. We only put 1 managed Microsoft.ML.FastTree.dll in our NuGet package.

We will either need to build the C# code twice, and include each in a RID-specific section of the NuGet package.

Or we will need to continue building the C# code once, and then have a switch at runtime whether we are running on ARM or not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I think that having the runtime switch is better. Instead of checking if we are running on arm or not I'll just check if the native dll is present or not. That way even blazer would work with this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I have added a runtime switch for this. It uses a delegate to pick between the native/managed version so it shouldn't have any performance impact, but if you could take a look I would appreciate it. I am gonig to run the benchmark tests before/after as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the benchmarks are basically identical speedwise before and after these changes. I had our benchmark run 30 times and the mean of the current code and the new code are within .15 seconds of each other on my local machine. So there shouldn't be noticeable performance impact from doing it this way.

<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<TargetsForTfmSpecificBuildOutput>$(TargetsForTfmSpecificBuildOutput);CopyProjectReferencesToPackage</TargetsForTfmSpecificBuildOutput>
</PropertyGroup>
Expand Down
2 changes: 1 addition & 1 deletion src/Native/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ function(install_library_and_symbols targetName)
endif()
endfunction()

if(NOT ${ARCHITECTURE} MATCHES "Arm64")
if(NOT ${ARCHITECTURE} MATCHES "arm.*")
add_subdirectory(CpuMathNative)
add_subdirectory(FastTreeNative)
add_subdirectory(MklProxyNative)
Expand Down
4 changes: 2 additions & 2 deletions src/Native/MatrixFactorizationNative/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@ project (MatrixFactorizationNative)
add_definitions(-D_SCL_SECURE_NO_WARNINGS)
add_definitions(-DUSEOMP)

if(NOT ${ARCHITECTURE} MATCHES "Arm64")
if(NOT ${ARCHITECTURE} MATCHES "arm.*")
add_definitions(-DUSESSE)
endif()

include_directories(libmf)

if(UNIX)
if(NOT ${ARCHITECTURE} MATCHES "Arm64")
if(NOT ${ARCHITECTURE} MATCHES "arm.*")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -O3 -pthread -std=c++0x -march=native")
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -O3 -pthread -std=c++0x")
Expand Down
16 changes: 8 additions & 8 deletions src/Native/Native.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,31 +82,31 @@
<!-- Copy MklImports into the native assets folder, so it can be treated like a regularly built native assembly - packaged into our NuGet packages, and used by tests. -->
<Message Text="PreparePackageAssets" Importance="High"/>

<Copy Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'"
<Copy Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'"
SourceFiles="$(NuGetPackageRoot)mlnetmkldeps\$(MlNetMklDepsPackageVersion)\runtimes\$(PackageRid)\native\$(NativeLibPrefix)MklImports$(NativeLibExtension)"
DestinationFolder="$(NativeAssetsBuiltPath)" />

<Copy Condition="'$(OS)' == 'Windows_NT' And '$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" SourceFiles="$(NuGetPackageRoot)mlnetmkldeps\$(MlNetMklDepsPackageVersion)\runtimes\$(PackageRid)\native\libiomp5md$(NativeLibExtension)"
<Copy Condition="'$(OS)' == 'Windows_NT' And '$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" SourceFiles="$(NuGetPackageRoot)mlnetmkldeps\$(MlNetMklDepsPackageVersion)\runtimes\$(PackageRid)\native\libiomp5md$(NativeLibExtension)"
DestinationFolder="$(NativeAssetsBuiltPath)" />

<ItemGroup>
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'"
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'"
Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)CpuMathNative$(NativeLibExtension)"
RelativePath="Microsoft.ML.CpuMath\runtimes\$(PackageRid)\nativeassets\netstandard2.0" />
<NativePackageAsset Include="$(PlaceholderFile)"
RelativePath="Microsoft.ML.CpuMath\runtimes\$(PackageRid)\nativeassets\netcoreapp3.1" />
<NativePackageAsset Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)LdaNative$(NativeLibExtension)"
RelativePath="Microsoft.ML\runtimes\$(PackageRid)\native" />
<!-- TODO: once we fix the 4 intel MKL methods, SymSgdNative will need to go back in. -->
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'"
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'"
Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)SymSgdNative$(NativeLibExtension)"
RelativePath="Microsoft.ML.Mkl.Components\runtimes\$(PackageRid)\native" />
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'"
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'"
Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)MklProxyNative$(NativeLibExtension)"
RelativePath="Microsoft.ML.Mkl.Redist\runtimes\$(PackageRid)\native" />
<NativePackageAsset Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)MatrixFactorizationNative$(NativeLibExtension)"
RelativePath="Microsoft.ML.Recommender\runtimes\$(PackageRid)\native" />
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'"
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'"
Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)FastTreeNative$(NativeLibExtension)"
RelativePath="Microsoft.ML.FastTree\runtimes\$(PackageRid)\native" />
</ItemGroup>
Expand All @@ -118,10 +118,10 @@
</ItemGroup>

<ItemGroup>
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'"
<NativePackageAsset Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'"
Include="$(NativeAssetsBuiltPath)\$(NativeLibPrefix)MklImports$(NativeLibExtension)"
RelativePath="Microsoft.ML.Mkl.Redist\runtimes\$(PackageRid)\native" />
<NativePackageAsset Condition="'$(OS)' == 'Windows_NT' And '$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="$(NativeAssetsBuiltPath)\libiomp5md$(NativeLibExtension)"
<NativePackageAsset Condition="'$(OS)' == 'Windows_NT' And '$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="$(NativeAssetsBuiltPath)\libiomp5md$(NativeLibExtension)"
RelativePath="Microsoft.ML.Mkl.Redist\runtimes\$(PackageRid)\native" />

</ItemGroup>
Expand Down
2 changes: 1 addition & 1 deletion src/Native/SymSgdNative/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ else()
endif()
endif()

if(NOT ${ARCHITECTURE} MATCHES "Arm64")
if(NOT ${ARCHITECTURE} MATCHES "arm.*")
find_library(MKL_LIBRARY MklImports HINTS ${MKL_LIB_PATH})
endif()

Expand Down
17 changes: 5 additions & 12 deletions src/Native/build.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,6 @@ exit /b 1
:: Setup vars for VS2019
set __PlatformToolset=v142
set __VSVersion=16 2019

:: Set the environment for the native build
call "%VS160COMNTOOLS%..\..\VC\Auxiliary\Build\vcvarsall.bat" %__VCBuildArch%

Expand All @@ -79,23 +78,17 @@ goto :SetupDirs
:: Setup vars for VS2017
set __PlatformToolset=v141
set __VSVersion=15 2017
if NOT "%__BuildArch%" == "arm64" (
if NOT "%__BuildArch%" == "arm" (
:: Set the environment for the native build
call "%VS150COMNTOOLS%..\..\VC\Auxiliary\Build\vcvarsall.bat" %__VCBuildArch%
)
)
:: Set the environment for the native build
call "%VS150COMNTOOLS%..\..\VC\Auxiliary\Build\vcvarsall.bat" %__VCBuildArch%

goto :SetupDirs

:VS2015
:: Setup vars for VS2015build
set __PlatformToolset=v140
set __VSVersion=14 2015
if NOT "%__BuildArch%" == "arm64" (
if NOT "%__BuildArch%" == "arm" (
:: Set the environment for the native build
call "%VS140COMNTOOLS%..\..\VC\vcvarsall.bat" %__VCBuildArch%
)
:: Set the environment for the native build
call "%VS140COMNTOOLS%..\..\VC\vcvarsall.bat" %__VCBuildArch%

:SetupDirs
:: Setup to cmake the native components
Expand Down
9 changes: 8 additions & 1 deletion src/Native/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ while [ "$1" != "" ]; do
--mkllibrpath)
shift
__mkllibrpath=$1
;;
;;
--stripsymbols)
__strip_argument="-DSTRIP_SYMBOLS=true"
;;
Expand Down Expand Up @@ -103,6 +103,13 @@ fi

__cmake_defines="${__cmake_defines} -DVERSION_FILE_PATH:STRING=${__versionSourceFile}"

OS=$(uname -m)

if [[ ( $OS == "amd64" || $OS == "x86_x64" ) && ( $__build_arch == "arm64" || $__build_arch == "arm" ) ]]; then
__cmake_defines="${__cmake_defines} -DCMAKE_TOOLCHAIN_FILE=$(PWD)/../../eng/common/cross/toolchain.cmake"
export TARGET_BUILD_ARCH=$__build_arch
fi

cd "$__IntermediatesDir"

echo "Building Machine Learning native components from $DIR to $(pwd)"
Expand Down
3 changes: 2 additions & 1 deletion src/Native/gen-buildsys-win.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ popd
:DoGen
if /i "%3" == "x64" (set __ExtraCmakeParams=%__ExtraCmakeParams% -A x64)
if /i "%3" == "x86" (set __ExtraCmakeParams=%__ExtraCmakeParams% -A Win32)
if /i "%3" == "Arm64" (set __ExtraCmakeParams=%__ExtraCmakeParams% -A Arm64)
if /i "%3" == "arm64" (set __ExtraCmakeParams=%__ExtraCmakeParams% -A arm64)
if /i "%3" == "arm" (set __ExtraCmakeParams=%__ExtraCmakeParams% -A arm)
"%CMakePath%" "-DCMAKE_BUILD_TYPE=%CMAKE_BUILD_TYPE%" "-DCMAKE_INSTALL_PREFIX=%__CMakeBinDir%" "-DMKL_LIB_PATH=%MKL_LIB_PATH%" "-DARCHITECTURE=%3" -G "Visual Studio %__VSString%" %__ExtraCmakeParams% -B. -H%1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need both -DARCHITECTURE and -A above? Can we just use 1 command line arg to set the archiecture?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the -A doesn't work on the Unix Makefile Generator, but that is how you set the architecture correctly for visual studio. I needed a way in the cmakelists file to be able to exclude native projects for both generators, and thats how I ended up with the -DARCHITECTURE. I'm sure there is a way I could use only the -A in visual studio and not need the -DARCHITECTURE, I just haven't been able to figure it out yet. We will always need -DARCHITECTURE for the Unix generator, its just if we can remove it from the visual studio one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you looked how it is done in other repos? For example dotnet/runtime?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm checking that out right now. They pass the -arch flag. Thats a custom flag. I'm still investigating how they plumb it internally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That eventually turns into this /p:TargetArchitecture=$arch. Still looking into how thats passed to the native side of things.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it actually looks like they do the same thing.
https://github.com/dotnet/runtime/blob/e4b4807e2fae2164d9116fbcdd49ba9044461e7e/eng/native/gen-buildsys.cmd#L34 they set the -A same way I do.

https://github.com/dotnet/runtime/blob/e4b4807e2fae2164d9116fbcdd49ba9044461e7e/src/coreclr/build-runtime.cmd#L450 they a flag -DCLR_CMAKE_TARGET_ARCH which is used the same why I am using the ARCHITECTURE flag. They do more complex stuff with it than we do since they target so many things, but its the same idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Thanks for verifying.

endlocal
GOTO :DONE
Expand Down
6 changes: 3 additions & 3 deletions test/Microsoft.Extensions.ML.Tests/Microsoft.Extensions.ML.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@
</ItemGroup>

<ItemGroup>
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="MklImports" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm' And '$(OS)' == 'Windows_NT'" Include="libiomp5md" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="MklImports" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm' And '$(OS)' == 'Windows_NT'" Include="libiomp5md" />
</ItemGroup>

<ItemGroup>
Expand Down
12 changes: 6 additions & 6 deletions test/Microsoft.ML.AutoML.Tests/Microsoft.ML.AutoML.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,14 @@

<!-- TODO: once we fix the 4 intel MKL methods, SymSgdNative will need to go back in. -->
<ItemGroup>
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="MklProxyNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'"Include="SymSgdNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="FastTreeNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="MklProxyNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="CpuMathNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="SymSgdNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="FastTreeNative" />
<NativeAssemblyReference Include="LdaNative" />
<NativeAssemblyReference Include="MatrixFactorizationNative" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm'" Include="MklImports" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'Arm64' And '$(TargetArchitecture)' != 'Arm' And '$(OS)' == 'Windows_NT'" Include="libiomp5md" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm'" Include="MklImports" />
<NativeAssemblyReference Condition="'$(TargetArchitecture)' != 'arm64' And '$(TargetArchitecture)' != 'arm' And '$(OS)' == 'Windows_NT'" Include="libiomp5md" />
</ItemGroup>

</Project>
Loading