Results 1 to 5 of 5

Thread: Recommend Commit time ?

  1. #1
    Junior Member
    Join Date
    Mar 2014
    Posts
    18

    Default Recommend Commit time ?

    What is the recommend Commit time for inserting large data-set (several GB)?

    Committing the changes upon each Insert makes the job so slow, so I've tried Commit at several level of data chunks, but all of them takes a meaningful of time (from minutes to even hours !!!).

    What's your recommendation of Commit time for small set of keys (1M) and very large values per key?

    A snapshot of memory usage during Commit ...



    Thanks.

    - Edit - 10:23 AM

    Beside that I noticed if I perform several Commits during the insert it will increase the total size of db in disk such that in a sample test if I commit it once the resulting db is 48 MB while if I perform 4 commits during the insert it would become 116 MB.

    Nearly 2.5 times larger .... Is that normal?
    Last edited by mafshin; 23.04.2014 at 08:58.

  2. #2

    Default

    In general, the storage engine commit does the following things:
    1. Stores modified W-tree nodes (without excluding them from cache).
    2. Flushes all scheme changes
    3. Stores the W-tree root
    4. Invokes IHeap.Commit() to stamp all changes in the backend file/stream.
    Of course, if the engine.Commit() is not invoked during long time insert, the engine will keep storing the modified W-tree nodes while they expire from cache. The stored data can not be guarantied until confirming with commit, but invoking commit after all will cause only the unstored nodes to be stored. After commit the back-end file is truncated as much as possible.

    Database Commit() is not fast operation, we recommend its usage as little as possible - after all, it is a global operation that has to guarantee the entire integrity.
    Last edited by a.todorov; 23.04.2014 at 19:54.

  3. #3

    Default Read performance during Commit()

    What happens to read performance while a Commit() is taking place? Are readers blocked, or do they read from the cache at usual speed?

  4. #4

    Default

    Quote Originally Posted by IanC View Post
    What happens to read performance while a Commit() is taking place? Are readers blocked, or do they read from the cache at usual speed?
    Please reply...

  5. #5

    Default

    Quote Originally Posted by IanC View Post
    What happens to read performance while a Commit() is taking place? Are readers blocked, or do they read from the cache at usual speed?
    Most of the time, the database tries to keep as little unsaved data as possible in the cache. But, if you read during commit operation, the performance will be slower.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
2002 - 2014 STS Soft SC. All Rights reserved.
STSdb, Waterfall Tree and WTree are registered trademarks of STS Soft SC.