LiteDb and SqliteDb problems

[haiduong87]

Description

I've encountered concurrency problem when using LiteDb or SqliteDb

Steps to Reproduce

1. Use any of two mentions caches
2. Deploy in IIS Web garden model (many workers)

Related code

insert short code snippets here

Expected behavior: [What you expected to happen]

Actual behavior: [What actually happened]

Specifications

• Provider : Latest LiteDb or SqliteDb
• Interceptor : AspectCore (version 0.2.0)
• Serializer : not use
• System : Windows 10 64bit, latest IIS

Problem with LiteDb

• LiteDB can deadlock itself, it seems. litedb-org/LiteDB#1863
• [QUESTION] Object synchronization method was called from an unsynchronized block of code litedb-org/LiteDB#1546
• Actually we can't handle this...

Problem with SqliteDb

• Null reference exception on DataTable.Load(IDataReader) dotnet/efcore#22756
• System.NullReferenceException at Microsoft.Data.Sqlite.SqliteConnection.RemoveCommand dotnet/efcore#20651
• It's about multi threads share one SqliteDbConnection
• My fix

`
public class SQLiteDatabaseProvider : ISQLiteDatabaseProvider
{
private static ConcurrentDictionary<int, SqliteConnection> _conns;

    /// <summary>
    ///     The builder
    /// </summary>
    private static SqliteConnectionStringBuilder _builder;
    /// <summary>
    ///     The conn.
    /// </summary>
    private static SqliteConnection _conn;

    /// <summary>
    ///     The options.
    /// </summary>
    private readonly SQLiteDBOptions _options;

    public SQLiteDatabaseProvider(string name, SQLiteOptions options)
    {
        DBProviderName = name;
        _options = options.DBConfig;
        _builder = new SqliteConnectionStringBuilder
        {
            DataSource = _options.DataSource,
            Mode = _options.OpenMode,
            Cache = _options.CacheMode
        };

        _conns = new ConcurrentDictionary<int, SqliteConnection>();
    }

    public string DBProviderName { get; } = EasyCachingConstValue.DefaultSQLiteName;

    /// <summary>
    ///     Gets the connection.
    /// </summary>
    /// <returns>The connection.</returns>
    public SqliteConnection GetConnection_Original()
    {
        if (_conn == null)
        {
            _conn = new SqliteConnection(_builder.ToString());
        }

        return _conn;
    }

    /// <summary>
    ///     Get or create the connection from pool (For thread safe)
    /// </summary>
    /// <returns></returns>
    public SqliteConnection GetConnection()
    {
        var threadId = Thread.CurrentThread.ManagedThreadId;
        var con = _conns.GetOrAdd(threadId, CreateNewConnection());
        Task.Run(async () =>
        {
            await Task.Delay(5000).ConfigureAwait(false);
            _conns.TryRemove(threadId, out var removingConn);
            if (removingConn?.State == ConnectionState.Closed)
            {
                removingConn.Dispose();
            }
        });
        return con;
    }

    private SqliteConnection CreateNewConnection()
    {
        return new SqliteConnection(_builder.ToString());
    }
}

/// <summary>
/// SQLiteCaching provider.
/// </summary>
public partial class DefaultSQLiteCachingProvider : EasyCachingAbstractProvider
{
    /// <summary>
    /// The cache.
    /// </summary>
    private ISQLiteDatabaseProvider _dbProvider;

    /// <summary>
    /// The options.
    /// </summary>
    private readonly SQLiteOptions _options;

    /// <summary>
    /// The logger.
    /// </summary>
    private readonly ILogger _logger;

    /// <summary>
    /// The cache.
    /// </summary>
    private SqliteConnection _cache => _dbProvider.GetConnection();
}    

`

[catcherwong]

@haiduong87 Thanks for your interest in this project.

We will take a look ASAP.

[catcherwong]

Based on your sqlite fixed code.

Some Unit tests do not pass yet.

[haiduong87]

Hi,

I'm not sure the test is stable :(
Just download master branch and try.

Note:

• I do several times
• With and without the fix
• Then delete the folder and try again with the original source (the captured picture is the original code)

[catcherwong]

When running the unit tests in a single class, all the tests can pass, but running together with 4 classes will not pass.

[catcherwong]

I have committed the code, and the unit tests of SQLite runs well in the CI environment.

It seems that there are some difference between using dotnet test and visual studio

A preview version was published for this issue.

BTW, PR is welcome, if you find out some problems, you can submit a PR to fix it. 😄

[haiduong87]

Hi,

I'm trying to fix the provider #264

But it does not pass all tests

[haiduong87]

Hi @catcherwong

I think I found the problem,

This static field is not good for "many providers" case.

And so the latest code has bug too.

I will make change and test again.