Page 4 of 4 FirstFirst ... 234
Results 31 to 34 of 34

Thread: STSDB Version 4

  1. #31

    Default

    STSdb W4.0 will be released at the end of March or in the first week of April.

  2. #32

    Default

    STSdb W4.0 release will be delayed. Non-development reasons.

  3. #33

    Default

    i want to ask : what's the diffrent whit 3 and 4?

  4. #34

    Default

    Main changes in STSdb W4.0 RC:

    1. New WaterfallTree™ data indexing structure:
    • real-time indexing
    • fast random key inserts
    • fast parallel inserts on multiple tables
    • outstanding performance on sequential keys
    • parallel engine architecture (works blazing on muliticore machines)

    2. New Garbage Collector
    • solved wasted space problems from STSdb R3.5
    • better reallocation capabilities
    • no data fragmentation

    3. No user types dependancies - and direct user type use

    The old STSdb R3.5 version was making storage data dependant from the user type.

    For each XTable
    XTable<TKey, TRecord> table;
    
    the engine serializes:
    • the IKeyMap<TKey> property (responsible to transform all TKey to/from byte[])
    • the IBinaryPersist<IIndexer<TRecord>> property (responsible to store the TRecord-s )

    This makes the database engaged with the TKey and TRecord types defined in the assembly and makes it almost impossible to work without these types.

    The new STSdb W4.0 version makes the storage data independent from the user types, while keep the possibility to work directly with it.

    The new version provides automatic transformation layer between user data and internal storage format.
        public interface IDataTransformer<T>
        {
            IData ToIData(T item);
            T FromIData(IData data);
            DataType DataType { get; }
        }
    
    For each user type T, the engine automatically generates an appropriate source code environment:
    • the equivalent IData type
    • IDataTransformer<T> logic
    • IComparer<IData> & IEqualityComparer<IData> methods
    • IData persist - responsible to store IData instances

    No matter of T complexity, the IData type and its roundabout logic is being created. The IData type is generated to be as close as it possible to the original type (it looks like an anonymous equivalent of T). The IComparer<IData> and IEqualityComparer<IData> logic is generated to be identical respectively to IComparer<T> and IEqualityComparer<T>.

    Thus, we work with IData instead of T.

    This makes the database independent from the user types while providing the users with the possibility to work directly with their custom types. The IData environment is generated with .NET expressions, so there is no performance penalty.

    4. Extended TKey & TRecord support

    STSdb R3.5 supports the following TKey & TRecord types:

    TKey types:
    • primitive .NET types:
    • TimeSpan
    • byte[]
    • structures (with value-type fields & StructLayoutAttribute.Value != LayoutKind.Auto)
    • Locator (specific for 3.5)
    • composite keys - XKey<T1, ..., T7> (where Tn is primitive)
    • any type with IKeyMap<TKey> implementation.

    TRecord types:
    • primitive .NET types
    • TimeSpan
    • byte[]
    • structures
    • enum types
    • Type
    • Image
    • Icon
    • MemoryStream
    • Guid
    • XElement
    • Locator (specific for 3.5)
    • BlobStream (specific for 3.5)
    • serializable objects
    • classes with public read/write properties of the above types (but only 1 level of nesting)
    • any type with IBinaryPersist<IIndexer<TRecord>> implementation.

    Because STSdb R3.5 engine uses custom radix tree, it requires keys of byte array only. This needs an additional map logic that converts each TKey to/from byte[] - the IKeyMap<TKey> implementation. This logic must be capable to convert each key to byte[] and back while keeping the exact original TKey order. This is not always easy to implement. Especially for the composite key types or ones with non-fixed length. Sometimes, even a map of a single key is hard to implement (for example: string and double). In other cases, the final byte[] key can be too long...

    Another inconvenience of 3.5 keys is that if the user wants to work with composite keys he must use the integrated XKey<T1,T2,..> types - he must write its own TCompositeKey <-> XKey<T1,T2,...> logic and use the XKey types in his code. That makes the code not so clean and elegant.

    In STSdb 4.0 there are no such limitations.

    The new version uses direct comparison method for each TKey type - it generates an appropriate IComparer<IData> logic, that looks еxactly as if the user would write it for TKey type.

    STSdb W4.0 RC supports the following TKey & TRecord types:
    • primitive .NET types
    • byte[]
    • enum types
    • structures and classes (with default constructor), containing public read/write properties with types from the above 3 categories
    • structures and classes (with default constructor), containing public read/write properties with types from the above 4 categories.

    Now we support nested types in both keys and records. And the big advantage here is the possibility to work directly with the TKey/TRecord types (almost no matter of its complexity) without losing performance. All details are handled by the expressions behind the scene.

    Note: We excluded some of the 3.5 types until the final architecture is refined. Moreover our plans are to make similar code-generated environments in other languages, so, for now, we will keep more common primitive type set.

    5. Parallel vertical compressions
    • uses .NET expressions to generate compression logic for each TKey/TRecord property.
    • works fast
    • has high compression ratio.

    6. Transactions
    • 3.5 supports ACID transactions.
    • 4.0 currently has a global commit at engine level only. (The final version will support ACID transactions.)

    7. Snapshots
    • 3.5 supports snapshots
    • 4.0 will support snapshots

    8. Thread-safe
    • 3.5 XTables are not thread-safe
    • 4.0 XIndexes are not thread-safe
    • 4.0 StorageEngine is thread-safe

    9. public API changes
    • XTable is now called XIndex. (We will keep the term XTable for the future releases, when we will support secondary indexes - then one XTable will manage miltiple XIndex instances. Until that, under the term 'table' we mean XIndex.)
    • no more Locator type in 4.0 (but still we can open the same table several times)
    • no Scheme table. (It it still a discussion whether there should be such metatable. Probably this will be clarified when the transaction model is being integrated.)

    10. Virtual File System
    • in 3.5 tables are placed in a separate virtual files and each table has its own radix tree
    • in 4.0 all tables are placed in one Waterfall tree. We keep the rule: many indexes - one W-tree, one W-tree - one storage device.
    • 4.0 has XFile - a virtual sparse file.
    • 4.0 has an abstract W-Tree layer on which can be built a system of tables and files

    11. 100% managed code
    • 3.5 has some unsafe pieces of code
    • 4.0 has no unsafe code.

    This is in brief. The hardest part in the whole development process was the research and development of the new data indexing structure. WaterfallTree took us almost two years. Now we have an effective solution for one of the toughest database problems.

    During the next few months we hope to complete the STSdb W4.0 roadmap and provide a complete high performance scalable database solution.

    STSdb W4.0 Roadmap:

    In RC2 version:
    • Client/Server support
    • .NET and Java connectors
    • support of public read/write fields in TKey/TRecord

    In Final version:
    • ACID transactions
    • snapshots
    • support of Array<T>, List<T>, Dictionary<K,V> as properties/fields in TKey/TRecord

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
2002 - 2014 STS Soft SC. All Rights reserved.
STSdb, Waterfall Tree and WTree are registered trademarks of STS Soft SC.