PredictionEnginePool.GetPredictionEngine is not thread safe

[strikene]

System information

• OS version/distro:Windows 10 Server 2019
• .NET Version (eg., dotnet --info): core 3.0

Issue

• What did you do?
  Invoked PredictionEnginePool.Predict("MyModelName", example) from multiple threads.
• What happened?
  Collection was modified; enumeration operation may not execute.
• What did you expect?
  Method is thread safe

Source code / logs

Package
Microsoft.Azure.Functions.Extensions Version="1.0.0"
Microsoft.Extensions.MLVersion="1.5.0-preview2" or 1.40
Microsoft.ML Version="1.5.0-preview2" or 1.40
Microsoft.NET.Sdk.Functions Version="3.0.3"

in startup

builder.Services.AddPredictionEnginePool<Input, PredictedEngineOutput>().FromFile("ModelName", "ModelPath", true);

in function
var re = _PredictionEnginePool.Predict("ModelName", new Input() {String = str });

Exception

   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source, Func`2 predicate)
   at Microsoft.Extensions.ML.PredictionEnginePoolPolicy`2.Return(PredictionEngine`2 obj)
   at Microsoft.Extensions.ObjectPool.DefaultObjectPool`1.Return(T obj)
   at Microsoft.Extensions.ML.PredictionEnginePool`2.ReturnPredictionEngine(String modelName, PredictionEngine`2 engine)
   at Microsoft.Extensions.ML.PredictionEnginePoolExtensions.Predict[TData,TPrediction](PredictionEnginePool`2 predictionEnginePool, String modelName, TData example)
   at BLDM_F.F.Predict(HttpRequest req, ILogger log) in C:\tfs\BLDM_F\Function1.cs:line 148

Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.

[antoniovs1029]

Thanks for reporting this. Can you please provide your complete code and dataset so that we can easily reproduce your issue? Thanks!

[strikene]

Thanks for reporting this. Can you please provide your complete code and dataset so that we can easily reproduce your issue? Thanks!

The attachment contains the source file and the model file. This problem occurs randomly and may take a long time to see. Thank you for your help
f.zip

[antoniovs1029]

Thanks. I am taking a look. Also, what kind of exception was thrown? It seems you only included the stacktrace on your description of the issue, but not the exception itself.

[strikene]

Thanks. I am taking a look. Also, what kind of exception was thrown? It seems you only included the stacktrace on your description of the issue, but not the exception itself.

Exception:Collection was modified; enumeration operation may not execute.

If I delete the following code, the exception will not appear, and I'm not sure about the correlation.

  #region get tag in model

        var pr = _PredictionEnginePool.Predict("m", new Input() { DMString = "ss" });
        var mo = _PredictionEnginePool.GetModel("m");
        var OutputSchema = _PredictionEnginePool.GetPredictionEngine("m").OutputSchema;
        var column = OutputSchema.GetColumnOrNull("Score");
        var slotNames = new VBuffer<ReadOnlyMemory<char>>();
        column.Value.GetSlotNames(ref slotNames);
        var alltag = slotNames.DenseValues().ToArray();
        TaginModel = new string[alltag.Count()];
        for (int i = 0; i < TaginModel.Length; i++)
        {
            TaginModel[i] = alltag[i].ToString();
        }
        #endregion

[antoniovs1029]

It seems the way List<WeakReference> _references on PredictionEnginePoolPolicy is being used is not thread-safe, so this is a bug.

@eerhardt Is looking into this now to fix it.