-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Microsoft.ML NuGet package doesn't work with packages.config #93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Hi @guivo, Can you give some more insight in how you referenced ML.NET? Did you use a Are you on the desktop .NET Framework? If so, you need to switch your project to be built for |
Hi @eerhardt , I have done the test creating a desktop C# project for a CLI application. I have then inluded Microsoft.ML using NuGet and the csproj is slightly different but I am not sure this is relevant:
More lines are there, to refence other ML modules (Api, Core, CpuMath....) I can confirm that I am targeting x64 (visual studio fails if try a different architecture). |
@guivo, Thanks for checking. Looks like the process fails to find the cpuMathNative.dll. Can you check that the file is in the same location as other managed DLLs and the exe? If it is not in the target directory, then it's likely a problem with our NuGet definition. To unblock yourself, you can copy that file there manually for now. Regards. |
@guivo - I get your issue now. The problem is that you are using https://blog.nuget.org/20180409/migrate-packages-config-to-package-reference.html You can also make |
Renaming the issue to best reflect the problem. Guidance from the NuGet team is:
|
@eerhardt migrating to PackageReference fixed the issue, thanks! Should I however expect a more flexible approach in the next release of the package? |
I haven't tried getting it to work, but it seems simple enough to fix Microsoft.ML to work with packages.config. So I'd hope we can fix it in the next release. |
Copy our native assemblies using MSBuild when a consumer is using NuGet packages.config, since NuGet doesn't do this automatically. Also, add an error when a consumer isn't targeting x64 to tell them ML.NET only supports x64. Fix dotnet#93
With regard to this thread, I would to mention that in the case a solution with 3 projects, the recipe I have used (Migrating to PackageReference) didn't have positive effect in the last project. Where the migration seemed successful but the runtime error about not finding CpuMathNative appears. Same code, in a solution with only that project, everything worked. |
Sorry, this wasn't actually an issue. It was a visual studio artifact (bug?): I asked to run an x86 conf but it was actually using anycpu, where the math library is not available. Thanks |
Copy our native assemblies using MSBuild when a consumer is using NuGet packages.config, since NuGet doesn't do this automatically. Also, add an error when a project is not targeting x64. ML.NET only supports x64. Fix #93
Copy our native assemblies using MSBuild when a consumer is using NuGet packages.config, since NuGet doesn't do this automatically. Also, add an error when a project is not targeting x64. ML.NET only supports x64. Fix dotnet#93
Same issue to me switcting to packagereference solved the question !! ;-) |
Reopening. We've regressed the packages.config support with #4096. If anyone runs into this with version
<Content Include="..\packages\Microsoft.ML.CpuMath.1.4.0\runtimes\win-x64\nativeassets\netstandard2.0\*.dll" Condition="'$(PlatformTarget)' == 'x64'">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Visible>false</Visible>
<Link>%(Filename)%(Extension)</Link>
</Content>
<Content Include="..\packages\Microsoft.ML.CpuMath.1.4.0\runtimes\win-x86\nativeassets\netstandard2.0\*.dll" Condition="'$(PlatformTarget)' == 'x86'">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Visible>false</Visible>
<Link>%(Filename)%(Extension)</Link>
</Content>
</ItemGroup> (NOTE: if your |
When we refactored CpuMath to support netcoreapp3.0, we broke the packages.config support to copy the native assembly. This fixes it again by copying the file from the correct location. Fix dotnet#93
commit a5e274ef8869576190bbb794360a5f56d998b470 Merge: b7db4fa d7f9996 Author: Keren Fuentes <[email protected]> Date: Thu Nov 14 14:51:21 2019 -0800 Merge branch 'onnx_bin_classifiers' of https://github.com/Lynx1820/machinelearning into onnx_bin_classifiers commit b7db4fa Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 17:41:12 2019 +0000 Added onnx export support for KeyToValueMappingTransformer (dotnet#4455) commit f3e0f6b Author: Eric Erhardt <[email protected]> Date: Thu Nov 14 07:22:12 2019 -0600 Fix a flaky Extensions.ML test. (dotnet#4458) * Fix a flaky Extensions.ML test. Make the reload model tests more resistant to timing changes. * PR feedback. commit c1e190a Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 05:24:14 2019 +0000 Added onnx export support for OptionalColumnTransform (dotnet#4454) * Initial work for adding onnx export support for OptionalColumnTransform * Implemented support for optional initializers in OnnxTranformer to support OptionalColumnTransform * Fixed handling of double values and non-long numeric types * Removed redundant line * Updated review comment commit f96761b Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 03:17:12 2019 +0000 Fixed model saving and loading of OneVersusAllTrainer to include SoftMax (dotnet#4472) * Fixed model saving and loading of OneVersusAllTrainer to include SoftMax * Modified existing test to include SoftMax option * Modified test to verify both cases: when UseSoftmax is true and false commit d45cc8a Author: Jake <[email protected]> Date: Wed Nov 13 17:26:49 2019 -0800 Add InternalsVisibleTo in AutoML and CodeGenerator for the assembly Microsoft.ML.ModelBuilder.AutoMLService.Gpu (dotnet#4474) commit 5e83e23 Author: Eric Erhardt <[email protected]> Date: Wed Nov 13 16:09:05 2019 -0600 CpuMathNative assembly is not getting copied when using packages.config. (dotnet#4465) When we refactored CpuMath to support netcoreapp3.0, we broke the packages.config support to copy the native assembly. This fixes it again by copying the file from the correct location. Fix dotnet#93 commit 693250b Author: Harish Kulkarni <[email protected]> Date: Wed Nov 13 21:58:07 2019 +0000 Added onnx export support for WordTokenizingTransformer and NgramExtractingTransformer (dotnet#4451) * Added onnx export support for string related transforms * Updated baseline test files A large portion of this commit is upgrading the baseline test files. The rest of the fixes deal with build breaks resulting from the upgrade of ORT version. * Fixed bugs in ValueToKeyMappingTransformer and added additional tests commit 5910910 Author: Antonio Velázquez <[email protected]> Date: Mon Nov 11 17:19:39 2019 -0800 Fixes dotnet#4292 about using PFI with BPT and CMPB (dotnet#4306) *Changes in PredictionTransformer.cs and Calibrator.cs to fix the problem of the create methods not being called, to make CMP load its internal calibrator and predictor first so to assign the correct paramaters types and runtimes, and added a PredictionTransformerLoadTypeAttribute so that the binary prediction transformer knows what type to assign when loading a CMP as its internal model. *Added a working sample for using PFI with BPT and CMPB while loading a model from disk. This is based entirely in the original sample. *Added file CalibratedModelParametersTests.cs with tests that the CMPs modified in this PR are now being correctly loaded from disk. *Changed a couple of tests in LbfgsTests.cs that failed because they used casts that now return 'null'. commit bcdac55 Author: Brian Stark <[email protected]> Date: Mon Nov 11 13:42:42 2019 -0800 Stabilize the LR test (dotnet#4446) * Stabilize the LR test Found issue with how we were using random for our ImageClassificationTrainer. This caused instability in our unit test, as we were not able to control the random seed. Modified the code to now use the same random object throughout, the trainer, thus allowing us to control the seed and therefor have predictable output. commit d7f9996 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:33:17 2019 -0800 workaround Scores commit 7fba31c Merge: 93388b6 c96d690 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:25:28 2019 -0800 merging changes commit 93388b6 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:19:59 2019 -0800 Added extraction of score column before node creation commit ea71828 Author: Keren Fuentes <[email protected]> Date: Fri Nov 8 15:53:11 2019 -0800 fix for binary classification trainers export to onnx commit 6fad293 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:26:43 2019 -0700 Revert "draft regression test" This reverts commit 1ad45c995516b9d39fc05aca855ce2abe96c407b. commit 83c1c80 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:24:23 2019 -0700 draft regression test commit 8884161 Author: frank-dong-ms <[email protected]> Date: Fri Nov 8 20:20:53 2019 -0800 nightly build pipeline (dotnet#4444) * nightly build pipeline commit c96d690 Author: Keren Fuentes <[email protected]> Date: Fri Nov 8 15:53:11 2019 -0800 fix for binary classification trainers export to onnx commit 8100364 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:26:43 2019 -0700 Revert "draft regression test" This reverts commit 1ad45c995516b9d39fc05aca855ce2abe96c407b. commit 81381e2 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:24:23 2019 -0700 draft regression test
The 1st Workaround was not alllowed so tried 2nd adding the code in .csproj but no effect.In Packages folder CpuMathNative.dll is seen but still the error continues.Dont know why it is not able to find. |
commit a5e274ef8869576190bbb794360a5f56d998b470 Merge: b7db4fa d7f9996 Author: Keren Fuentes <[email protected]> Date: Thu Nov 14 14:51:21 2019 -0800 Merge branch 'onnx_bin_classifiers' of https://github.com/Lynx1820/machinelearning into onnx_bin_classifiers commit b7db4fa Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 17:41:12 2019 +0000 Added onnx export support for KeyToValueMappingTransformer (dotnet#4455) commit f3e0f6b Author: Eric Erhardt <[email protected]> Date: Thu Nov 14 07:22:12 2019 -0600 Fix a flaky Extensions.ML test. (dotnet#4458) * Fix a flaky Extensions.ML test. Make the reload model tests more resistant to timing changes. * PR feedback. commit c1e190a Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 05:24:14 2019 +0000 Added onnx export support for OptionalColumnTransform (dotnet#4454) * Initial work for adding onnx export support for OptionalColumnTransform * Implemented support for optional initializers in OnnxTranformer to support OptionalColumnTransform * Fixed handling of double values and non-long numeric types * Removed redundant line * Updated review comment commit f96761b Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 03:17:12 2019 +0000 Fixed model saving and loading of OneVersusAllTrainer to include SoftMax (dotnet#4472) * Fixed model saving and loading of OneVersusAllTrainer to include SoftMax * Modified existing test to include SoftMax option * Modified test to verify both cases: when UseSoftmax is true and false commit d45cc8a Author: Jake <[email protected]> Date: Wed Nov 13 17:26:49 2019 -0800 Add InternalsVisibleTo in AutoML and CodeGenerator for the assembly Microsoft.ML.ModelBuilder.AutoMLService.Gpu (dotnet#4474) commit 5e83e23 Author: Eric Erhardt <[email protected]> Date: Wed Nov 13 16:09:05 2019 -0600 CpuMathNative assembly is not getting copied when using packages.config. (dotnet#4465) When we refactored CpuMath to support netcoreapp3.0, we broke the packages.config support to copy the native assembly. This fixes it again by copying the file from the correct location. Fix dotnet#93 commit 693250b Author: Harish Kulkarni <[email protected]> Date: Wed Nov 13 21:58:07 2019 +0000 Added onnx export support for WordTokenizingTransformer and NgramExtractingTransformer (dotnet#4451) * Added onnx export support for string related transforms * Updated baseline test files A large portion of this commit is upgrading the baseline test files. The rest of the fixes deal with build breaks resulting from the upgrade of ORT version. * Fixed bugs in ValueToKeyMappingTransformer and added additional tests commit 5910910 Author: Antonio Velázquez <[email protected]> Date: Mon Nov 11 17:19:39 2019 -0800 Fixes dotnet#4292 about using PFI with BPT and CMPB (dotnet#4306) *Changes in PredictionTransformer.cs and Calibrator.cs to fix the problem of the create methods not being called, to make CMP load its internal calibrator and predictor first so to assign the correct paramaters types and runtimes, and added a PredictionTransformerLoadTypeAttribute so that the binary prediction transformer knows what type to assign when loading a CMP as its internal model. *Added a working sample for using PFI with BPT and CMPB while loading a model from disk. This is based entirely in the original sample. *Added file CalibratedModelParametersTests.cs with tests that the CMPs modified in this PR are now being correctly loaded from disk. *Changed a couple of tests in LbfgsTests.cs that failed because they used casts that now return 'null'. commit bcdac55 Author: Brian Stark <[email protected]> Date: Mon Nov 11 13:42:42 2019 -0800 Stabilize the LR test (dotnet#4446) * Stabilize the LR test Found issue with how we were using random for our ImageClassificationTrainer. This caused instability in our unit test, as we were not able to control the random seed. Modified the code to now use the same random object throughout, the trainer, thus allowing us to control the seed and therefor have predictable output. commit d7f9996 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:33:17 2019 -0800 workaround Scores commit 7fba31c Merge: 93388b6 c96d690 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:25:28 2019 -0800 merging changes commit 93388b6 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:19:59 2019 -0800 Added extraction of score column before node creation commit ea71828 Author: Keren Fuentes <[email protected]> Date: Fri Nov 8 15:53:11 2019 -0800 fix for binary classification trainers export to onnx commit 6fad293 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:26:43 2019 -0700 Revert "draft regression test" This reverts commit 1ad45c995516b9d39fc05aca855ce2abe96c407b. commit 83c1c80 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:24:23 2019 -0700 draft regression test commit 8884161 Author: frank-dong-ms <[email protected]> Date: Fri Nov 8 20:20:53 2019 -0800 nightly build pipeline (dotnet#4444) * nightly build pipeline commit c96d690 Author: Keren Fuentes <[email protected]> Date: Fri Nov 8 15:53:11 2019 -0800 fix for binary classification trainers export to onnx commit 8100364 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:26:43 2019 -0700 Revert "draft regression test" This reverts commit 1ad45c995516b9d39fc05aca855ce2abe96c407b. commit 81381e2 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:24:23 2019 -0700 draft regression test
* Squashed commit of the following: commit a5e274ef8869576190bbb794360a5f56d998b470 Merge: b7db4fa d7f9996 Author: Keren Fuentes <[email protected]> Date: Thu Nov 14 14:51:21 2019 -0800 Merge branch 'onnx_bin_classifiers' of https://github.com/Lynx1820/machinelearning into onnx_bin_classifiers commit b7db4fa Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 17:41:12 2019 +0000 Added onnx export support for KeyToValueMappingTransformer (#4455) commit f3e0f6b Author: Eric Erhardt <[email protected]> Date: Thu Nov 14 07:22:12 2019 -0600 Fix a flaky Extensions.ML test. (#4458) * Fix a flaky Extensions.ML test. Make the reload model tests more resistant to timing changes. * PR feedback. commit c1e190a Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 05:24:14 2019 +0000 Added onnx export support for OptionalColumnTransform (#4454) * Initial work for adding onnx export support for OptionalColumnTransform * Implemented support for optional initializers in OnnxTranformer to support OptionalColumnTransform * Fixed handling of double values and non-long numeric types * Removed redundant line * Updated review comment commit f96761b Author: Harish Kulkarni <[email protected]> Date: Thu Nov 14 03:17:12 2019 +0000 Fixed model saving and loading of OneVersusAllTrainer to include SoftMax (#4472) * Fixed model saving and loading of OneVersusAllTrainer to include SoftMax * Modified existing test to include SoftMax option * Modified test to verify both cases: when UseSoftmax is true and false commit d45cc8a Author: Jake <[email protected]> Date: Wed Nov 13 17:26:49 2019 -0800 Add InternalsVisibleTo in AutoML and CodeGenerator for the assembly Microsoft.ML.ModelBuilder.AutoMLService.Gpu (#4474) commit 5e83e23 Author: Eric Erhardt <[email protected]> Date: Wed Nov 13 16:09:05 2019 -0600 CpuMathNative assembly is not getting copied when using packages.config. (#4465) When we refactored CpuMath to support netcoreapp3.0, we broke the packages.config support to copy the native assembly. This fixes it again by copying the file from the correct location. Fix #93 commit 693250b Author: Harish Kulkarni <[email protected]> Date: Wed Nov 13 21:58:07 2019 +0000 Added onnx export support for WordTokenizingTransformer and NgramExtractingTransformer (#4451) * Added onnx export support for string related transforms * Updated baseline test files A large portion of this commit is upgrading the baseline test files. The rest of the fixes deal with build breaks resulting from the upgrade of ORT version. * Fixed bugs in ValueToKeyMappingTransformer and added additional tests commit 5910910 Author: Antonio Velázquez <[email protected]> Date: Mon Nov 11 17:19:39 2019 -0800 Fixes #4292 about using PFI with BPT and CMPB (#4306) *Changes in PredictionTransformer.cs and Calibrator.cs to fix the problem of the create methods not being called, to make CMP load its internal calibrator and predictor first so to assign the correct paramaters types and runtimes, and added a PredictionTransformerLoadTypeAttribute so that the binary prediction transformer knows what type to assign when loading a CMP as its internal model. *Added a working sample for using PFI with BPT and CMPB while loading a model from disk. This is based entirely in the original sample. *Added file CalibratedModelParametersTests.cs with tests that the CMPs modified in this PR are now being correctly loaded from disk. *Changed a couple of tests in LbfgsTests.cs that failed because they used casts that now return 'null'. commit bcdac55 Author: Brian Stark <[email protected]> Date: Mon Nov 11 13:42:42 2019 -0800 Stabilize the LR test (#4446) * Stabilize the LR test Found issue with how we were using random for our ImageClassificationTrainer. This caused instability in our unit test, as we were not able to control the random seed. Modified the code to now use the same random object throughout, the trainer, thus allowing us to control the seed and therefor have predictable output. commit d7f9996 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:33:17 2019 -0800 workaround Scores commit 7fba31c Merge: 93388b6 c96d690 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:25:28 2019 -0800 merging changes commit 93388b6 Author: Keren Fuentes <[email protected]> Date: Mon Nov 11 11:19:59 2019 -0800 Added extraction of score column before node creation commit ea71828 Author: Keren Fuentes <[email protected]> Date: Fri Nov 8 15:53:11 2019 -0800 fix for binary classification trainers export to onnx commit 6fad293 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:26:43 2019 -0700 Revert "draft regression test" This reverts commit 1ad45c995516b9d39fc05aca855ce2abe96c407b. commit 83c1c80 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:24:23 2019 -0700 draft regression test commit 8884161 Author: frank-dong-ms <[email protected]> Date: Fri Nov 8 20:20:53 2019 -0800 nightly build pipeline (#4444) * nightly build pipeline commit c96d690 Author: Keren Fuentes <[email protected]> Date: Fri Nov 8 15:53:11 2019 -0800 fix for binary classification trainers export to onnx commit 8100364 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:26:43 2019 -0700 Revert "draft regression test" This reverts commit 1ad45c995516b9d39fc05aca855ce2abe96c407b. commit 81381e2 Author: Keren Fuentes <[email protected]> Date: Thu Oct 31 15:24:23 2019 -0700 draft regression test * update name and remove some dependency code * baseline file changes * Refactored function
I am getting the same error but on different scenario. I have 2 input fields one is string and one is int/float. If I am setting only text attribute to my model it is working fine, but when I set int its giving that Microsoft.ML.CpuMathNatiive.dll not loaded error. I am running on x64 mode using ML 1.4.0 I am trying to consume this model in a service and getting this error. |
@samcode22 Is your issue solved? |
@coderdipti - can you try with the latest pre-release version of ML.NET? https://www.nuget.org/packages/Microsoft.ML/1.5.0-preview. If that doesn't fix your issue, please open a new issue to track it with full repro steps - https://github.com/dotnet/machinelearning/issues/new |
I think the problem will be solved in the next version, temporarily type |
Hi everyone |
@cemilsavtekin - this issue is quite old. Could you open an new issue with as much information as you can? Useful information would be:
|
Original Title
Impossible to run Iris classification test
System information
Windows 10.0.17134
Product Information:
Version: 2.1.200
Commit SHA-1 hash: 2edba8d7f1
Issue
What did you do?
Try Iris tutorial
What happened?
Runtime error:
System.Reflection.TargetInvocationException
HResult=0x80131604
Message=Eccezione generata dalla destinazione di una chiamata.
Source=mscorlib
StackTrace:
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.ML.Runtime.EntryPoints.EntryPointNode.Run()
at Microsoft.ML.Runtime.EntryPoints.EntryPointGraph.RunNode(EntryPointNode node)
at Microsoft.ML.Runtime.EntryPoints.JsonUtils.GraphRunner.RunAllNonMacros()
at Microsoft.ML.Runtime.EntryPoints.JsonUtils.GraphRunner.RunAll()
at Microsoft.ML.LearningPipeline.TrainTInput,TOutput
at IrsiExample.Program.Main(String[] args) in C:\Users\gvolpi\source\repos\MLNetTest1\IrsiExample\Program.cs:line 69
Inner Exception 1:
TargetInvocationException: Eccezione generata dalla destinazione di una chiamata.
Inner Exception 2:
DllNotFoundException: Impossibile caricare la DLL 'CpuMathNative': Impossibile trovare il modulo specificato. (Eccezione da HRESULT: 0x8007007E).
Source code / logs
Code copy/pasted from github
The text was updated successfully, but these errors were encountered: