Sunday, October 21, 2018



There is also another benefit to the full-text search. We are not restricted to their import into any form of storage. Object Storage can serve as the source for all databases including graph databases. There is generally a lot of preparation when data is exported from relational tables and imported into the graph databases when theoretically all the relations in the relational tables are merely edges to the nodes representing the entities. Graph databases are called natural databases because the relationships can be enumerated and persisted as edges but it is this enumeration that takes some iterations.  Data extract transform and load operations have rigorous packages in the relational world and largely relying on the consistency checks but they are not the same in the graph database. Therefore, each operation requires validation and more so when an organization is importing the data into a graph database without precedent. The indexer documents overcome the import because the data does not need to be collected. The inverted list of documents is easy to compare for Intersection, left and right differences and they add to edge weights directly when the terms are treated as nodes.   The ease with which data can be viewed as nodes and edges makes the import easier. In this way, the object storage for indexer provides convenience to destinations such as graph database where the inverted list of documents may be used in graph algorithms.
Full-text search is not the only stack. There can be another search stack that can be added to this object storage. For example, an iterator-based .NET style standard query operator may also be provided over this object storage. Even query tools like LogParser that opens up a COM interface to the objects in the storage can be used. Finally, a comprehensive and dedicated query engine that studies, caches and replays the query is possible since the object storage does not restrict the search layer.
There are a few techniques that can improve query execution. The degree of parallelism helps the query to execute faster by partitioning the data and invoking multiple threads. while increasing these parallel activities, we should be careful to not have too many otherwise the system can get into thread thrashing mode. The rule of thumb for increasing DoP is that the number of threads is one more than the number of processors and this refers to the operating system threads. There are no limits to lightweight workers  that do not have contention.
Caching is another benefit to query execution. if a query repeats itself over and over again, we need not perform the same calculations to determine the least cost of serving the query. We can cache the plan and the costs which we can reuse.


No comments:

Post a Comment