Results 1 to 5 of 5

Thread: Parallel write into two different tables error. STSdb 3.5.10 unreleased

  1. #1

    Default Parallel write into two different tables error. STSdb 3.5.10 unreleased

    Parallel threads write into two different tables sometimes (very often) brings to different mistakes:
    HTML Code:
    STSdb.MTL: NATIVE COMMIT FAILED System.ArgumentException: index.Length != Depth
       bei STSdb.Data.FTable`2.set_Item(Byte[] index, TRecord value) in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Data\FTable.cs:Zeile 114.
       bei STSdb.Data.FTable`2.set_Item(TKey key, TRecord value) in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Data\FTable.cs:Zeile 132.
       bei STSdb.Data.XTable`2.Commit() in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Data\XTable.cs:Zeile 1311.
       bei STSdb.Data.Transaction.Commit() in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Data\Transaction.cs:Zeile 84.
       bei STSdb.MTL.TransactionsCoordinator.Commit(Int32 transactionThreadId) in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\MTL\TransactionsCoordinator.cs:Zeile 451.
    or
    HTML Code:
     STSdb.MTL: NATIVE COMMIT FAILED System.ArgumentNullException: Der Wert darf nicht NULL sein.
    Parametername: type
       bei System.Xml.Serialization.XmlSerializer..ctor(Type type, String defaultNamespace)
       bei STSdb.General.Streams.PrimitiveSerialization.ReadAsOjbect(BinaryReader reader) in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\General\Streams\PrimitiveSerialization.cs:Zeile 249.
       bei STSdb.Files.BinaryFile.ReadObject() in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Files\BinaryFile.cs:Zeile 222.
       bei STSdb.Data.FTable`2.LoadHeader(Byte activeHeader) in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Data\FTable.cs:Zeile 928.
       bei STSdb.Data.FTable`2..ctor(RawFile rawFile, UInt64 offset) in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Data\FTable.cs:Zeile 72.
       bei STSdb.Data.XTable`2.Commit() in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Data\XTable.cs:Zeile 1307.
       bei STSdb.Data.Transaction.Commit() in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\Data\Transaction.cs:Zeile 102.
       bei STSdb.MTL.TransactionsCoordinator.Commit(Int32 transactionThreadId) in C:\Users\blaze\Documents\Visual Studio 2010\Projects\tests\STSDBrev3900_on_codeplex\MTL\TransactionsCoordinator.cs:Zeile 451.
    Sometimes DB is not readable anymore.

    To test the issue you can download the solution from here STSdb Multi-Threading Layer Change Set 9902 and press Button1 on the start form.

    Is it possible to write into two different tables in parallel, without synchronization, at all?
    Last edited by blaze; 09.12.2011 at 13:18.

  2. #2

    Default

    So, seems to be, inserts in different tables locators can be done in parallel, but Commit (or Rollback) for all parallel threads must come in a serial sequence, so locked. Issue is closed.

    Some tests:

    IDE HDD test: 20 threads, every inserts 1.000.000 records, every of record contains
    120 UTF-8 symbols. Total Db-size: 234 MB. Insert time for all threads 40 sec.
    Speed around 6MB/s

  3. #3

    Default

    Some more tests:

    IDE HDD test: 20 threads, every inserts 1.000.000 records, every of record contains
    600 UTF-8 symbols. Total Db-size: 1.063 MB. Insert time for all threads 141 sec.
    Speed is around 7.5MB/s...

    Interesting, is it possible to make it faster?

  4. #4

    Default

    Interesting results.

    Try to write a custom persist layer. This usually doubles the insert speed.

  5. #5

    Default

    Is it possible to write into two different tables in parallel, without synchronization, at all?
    1. XTable instances are not thread safe. If you want to write into a single table from different threads, you have to access all methods and properties through a synchronization object:
    lock (table.SyncRoot)
        table[key] = record;
    
    The table SyncRoot object is actually the synchronization object of the backend RawFile.

    2. If you want to write into different tables from a different threads, you have to synchronize only the creating of the tables (i.e. registering to the scheme):
    lock (engine.Scheme.SyncRoot)
        table = engine.Scheme.CreateOrOpenXTable<TKey, TRecord>(locator);
    

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
2002 - 2014 STS Soft SC. All Rights reserved.
STSdb, Waterfall Tree and WTree are registered trademarks of STS Soft SC.