Saturday, August 2, 2014

Splunk Search Processors are generally of the following types:

Streaming

This is completely parallelizable with example operators as eval, where, rex

Stateful

This requires a single stream of data and therefore is not parallelizable. Example commands are sort, eventstats, etc.
Event-processing
This is not reporting, but also not streaming or stateful. Example commands are : Sort, eventstats etc.

Reporting

This is summarizing. Example commands are : stats, chart, timechart, top etc.
For now, we can exclude sub-searches.
Searches can be saved so we will need a SavedSearchAdminHandler that implements similar methods as for an AdminHandler. 

Batched search:


            Batch mode is not realtime and not summarizing. It is a reporting search such as the stats command. It operates one bucket at a time. Buckets are sorted into non-restartable and restartable buckets and read in that order – a few events at a time until the bucket gets exhausted.

Realtime search:


            Realtime searches yield results as the events are indexed, even in a distributed environment. SplunkLite.Net is a single machine single instance mode and nothing stops it from being distributable especially since there’s CCW and REST support in .Net. This has the ability to do search and stats on the incoming pipeline data.

Indexed search: 

This works on different time windows and adjust the time based on user preferences . For example – last one week, one month, one year etc.  Buckets may overlap in time although only a few buckets will be writeable (hot). This kind of search runs with map/reduce like distribution.

For each of the searches it supports three different modes.
-       Fast mode
o   Field discovery off for event searches. No event or field data for stats searches.
-       Smart mode
o   Field discovery on for event searches. No event or field data for stats searches.
-       Verbose mode
o   All event and field data

No comments:

Post a Comment