Race condition with named model pool initialization.

[MCPx]

System information

• OS version/distro: Windows
• .NET Version (eg., dotnet --info): .NET Core 3.1

Issue

• What did you do?

Changed my call to AddPredictionEnginePool in my ASP.net Core app to use a model name and updated my Predict function to load the named model.

• What happened?

Calling the API with more than one prediction request at once started causing the API to crash.

• What did you expect?

The PredictionEnginePool should have been able to handle multiple requests at once, as it did with the default pool. Adding a call on API startup to get the prediction engine for that name and returning works as a workaround (or adding a lock around the Predict call), but should the named pools not be initialized on startup like the default pool is? Or provide the option to perform that initialization automatically or warn in the documentation that the named pools aren't thread safe on the first call.

Source code / logs

Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.

[antoniovs1029]

Wonder if this is related to #4981 which got fixed on #4992 . Notice that the fix has just been included inside the ML.NET version 1.5.0 release (which got released last week).

[MCPx]

Hi @antoniovs1029, this issue was encountered on version 1.5.0 (and Microsoft.Extensions.ML version 1.5.0). I also did not get an exception, like in that issue. The API just crashes and restarts when it hits the Predict function.

[frank-dong-ms-zz]

@MCPx can you please provide a repro project if possible that can help us repro the issue?

[Lynx1820]

Hi @MCPx ,

I tried load testing by making a script with 100 sequential inference calls, and running the script twice, in parallel. I did not encounter a crash. If you can, please provide a repro. Thank you.

[mstfbl]

Hi @MCPx ,

@Lynx1820 has attempted to reproduce the race condition crash you've obtained, and has not encountered an error. We don't have any further means of reproducing your error. Can you provide a reproducible project and/or code for your pipeline with which you've obtained this issue?

[harishsk]

Hi @MCPx Can you please reopen the issue when you can share some sample code & data that we can use to reproduce the issue?