In today's post we continue our discussion on porting  the Splunk forwarder functionality to Splunk Lite.Net Here we look at the data structures such as the QueueReader and QueueWriter in ProducerConsumerQueue. We are making sure the ProducerConsumerQueue has equivalent in.Net
Queue is the abstact notion to read CowPipelineData.
QueueReader is the interface for the client to read data from the queue
QueueWriter is the interface for the client app to add data to the queue. There are several queue implementations
Each QueueWriterImplementations has additional methods to get the size, check if the queue is full or empty, reset the size, get stats, remove a single item, add metrics and shutdown.
The writers can be in memory or in disk writers.
A PersistentQueue is one which has one reader and many writers. The writers write to memory but when they overflow they right to disk.
A ProducerConsumerQueue has in memory writers by name, persistent writers by name, list of all writers, the next provider to be polled for data, a mutex protecting condition, and a queue not empty condition.
 With the PipelineData,we mentioned earlier that each thread can have its exlusive CowPipelineData though underlying they may share the same PipelineData.
Moreover, we mentioned that the PipelineData uses notion of RawStoragePool and RawStorageReference but these are not suitable for porting because we don't manage memory in .Net.
Anytime the underlying data is modified, we do something like this in the forwarder:
PipelineData *x  = pData.getWriter()
x->put(...)
Together this shows that we can use .Net collection but retain the semantics of a queue reader and writer in SplunkLite.Net.
Search is one of the biggest operations and we will deal with that soon.