ML.FastTree nuget package doesn't work with packages.config

[eerhardt]

Try using Microsoft.ML.FastTree on .NET Framework using packages.config.

You get an exception:

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.DllNotFoundException: Unable to load DLL 'FastTreeNative': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
   at Microsoft.ML.Trainers.FastTree.DenseIntArray.C_Sumup_double(Int32 numBits, Byte* pData, Int32* pIndices, Double* pSampleOutputs, Double* pSampleOutputWeights, Double* pSumTargetsByBin, Double* pSumTargets2ByBin, Int32* pCountByBin, Int32 totalCount, Double totalSampleOutputs, Double totalSampleOutputWeights)
   at Microsoft.ML.Trainers.FastTree.DenseIntArray.SumupCPlusPlusDense(SumupInputData input, FeatureHistogram histogram, Byte* data, Int32 numBits)
   at Microsoft.ML.Trainers.FastTree.Dense8BitIntArray.Sumup(SumupInputData input, FeatureHistogram histogram)
   at Microsoft.ML.Trainers.FastTree.FeatureHistogram.SumupWeighted(Int32 numDocsInLeaf, Double sumTargets, Double sumWeights, Double[] outputs, Double[] weights, Int32[] docIndices)

This is because we aren't copying the FastTreeNative.dll to the output folder.

We should be including https://github.com/dotnet/machinelearning/blob/master/pkg/common/CommonPackage.props in the nuget package, and this would start working.

@shauheen @glebuk @TomFinley

[shauheen]

Thanks @eerhardt for filing and fixing :)

[alex-resdiary]

I am using ML.Net.FastTree and i keep running into this issue even after re-installing the package :
Unable to load DLL 'FastTreeNative': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

Any ideas on how to solve this issue?

[eerhardt]

@alex-resdiary - could you give a little more information on your project? (If you could zip up your project and send it to me, that would be easiest. My email is in my github profile.)

Are you targeting .NET Framework or .NET Core? Are you using packages.config? If you are targeting .NET Framework and packages.config, are you building for x64?

[jtsoftware]

I'm seeing the same problem.
.NET Framework 4.7.2
ML .NET 1.7.9
Using package.config
All CPUs or x64.

I'll send you a repro solution.

The exception info:

System.AggregateException
HResult=0x80131500
Message=One or more errors occurred.
Source=mscorlib
StackTrace:
at System.Threading.Tasks.Parallel.Invoke(ParallelOptions parallelOptions, Action[] actions)
at Microsoft.ML.Trainers.FastTree.ThreadTaskManager.ThreadTask.RunTask()
at Microsoft.ML.Trainers.FastTree.LeastSquaresRegressionTreeLearner.FindBestSplitOfRoot(Double[] targets)
at Microsoft.ML.Trainers.FastTree.LeastSquaresRegressionTreeLearner.FitTargets(IChannel ch, Boolean[] activeFeatures, Double[] targets)
at Microsoft.ML.Trainers.FastTree.RandomForestLeastSquaresTreeLearner.FitTargets(IChannel ch, Boolean[] activeFeatures, Double[] weightedtargets, Double[] targets, Double[] weights)
at Microsoft.ML.Trainers.FastTree.RandomForestOptimizer.TrainingIteration(IChannel ch, Boolean[] activeFeatures)
at Microsoft.ML.Trainers.FastTree.FastTreeTrainerBase3.Train(IChannel ch) at Microsoft.ML.Trainers.FastTree.FastTreeTrainerBase3.TrainCore(IChannel ch)
at Microsoft.ML.Trainers.FastTree.FastForestBinaryTrainer.TrainModelCore(TrainContext context)
at Microsoft.ML.Trainers.TrainerEstimatorBase2.TrainTransformer(IDataView trainSet, IDataView validationSet, IPredictor initPredictor) at Microsoft.ML.Trainers.TrainerEstimatorBase2.Fit(IDataView input)
at Microsoft.ML.Data.EstimatorChain1.Fit(IDataView input) at MTCWordPredictionModel.Base.ModelTrainerBase.Fit(String inputFilePath) in F:\Repros\FatTreeNativeProblem\MTCWordPredictionModel\Base\ModelTrainerBase.cs:line 62 at MTCWordPredictionModel.Test.LocalModelTest.Train(IModelTrainerBase trainer) in F:\Repros\FatTreeNativeProblem\MTCWordPredictionModel\Test\LocalModelTest.cs:line 59 at MTCWordPredictionModel.Test.LocalModelTest.<RunTest>b__17_0(IModelTrainerBase t) in F:\Repros\FatTreeNativeProblem\MTCWordPredictionModel\Test\LocalModelTest.cs:line 48 at System.Collections.Generic.List1.ForEach(Action`1 action)
at MTCWordPredictionModel.Test.LocalModelTest.RunTest(String trainingFilePath, String testFilePath, String outputFilePath, String modelFilePath) in F:\Repros\FatTreeNativeProblem\MTCWordPredictionModel\Test\LocalModelTest.cs:line 48
at FatTreeNativeProblem.Program.Main(String[] args) in F:\Repros\FatTreeNativeProblem\FatTreeNativeProblem\Program.cs:line 21

This exception was originally thrown at this call stack:
[External Code]

Inner Exception 1:
DllNotFoundException: Unable to load DLL 'FastTreeNative': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

[eerhardt]

Is the .dll in the output folder?

[jtsoftware]

Is the .dll in the output folder?

No. But if I copy "F:\Repros\FatTreeNativeProblem\packages\Microsoft.ML.FastTree.1.7.0\runtimes\win-x86\native\FastTreeNative.dll" to bin\Debug, I don't get the error. Shouldn't this be copied by NuGet when I install Microsoft.ML.FastTree? I have to copy it manually. If I try to add the above file as a reference, I get an error from Visual Studio. The solution I emailed you demonstrates this. I also tried uninstalling and reinstalling the FastTree package.

…

On Thu, Jan 20, 2022 at 11:40 AM Eric Erhardt ***@***.***> wrote: Is the .dll in the output folder? — Reply to this email directly, view it on GitHub <#3626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOW64JIXGJBKKDZWJSA5T3UXBQJZANCNFSM4HJQMOJQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

-- John Thompson ***@***.*** https://www.jtlanguage.com 1-909-283-4364 (home) 1-909-283-5642 (cell)

[eerhardt]

The issue with your project is that you are using packages.config and a ProjectReference from your "app" project to your "library" project. With packages.config, the NuGet dependencies aren't transitive, like they are with PackageReference. My suggestion is to convert your project to use PackageReference instead of packages.config. See https://docs.microsoft.com/en-us/nuget/consume-packages/migrate-packages-config-to-package-reference.

Alternatively, if you want to continue using packages.config, you need to install the Microsoft.ML.FastTree package in the "app"/ "exe" project.

[jtsoftware]

Thanks. Got it. I thought Visual Studio copied dependent DLLs, but I guess the FastTreeNative.dll is loaded dynamically, so VS didn't know about it.