We were discussing Riverbed Operating System (RiOS) technical concepts. We know that RiOS can accelerate TCP based connections. These include but are not limited to CIFS, NFS, FTP, HTTP, HTTPS and database connections. Data Streamlining reduces bandwidth consumption by 60 to 95 %.
We now look at the specifics of data de-duplication. RiOS intercepts and analyzes TCP traffic, segmenting the data and indexing it. The indexed data is resident on disk and can be compared. If a segment of data has been seen before, it is not transferred across the WAN, instead a reference is sent in its place. This process  enables duplicate data to be replaced by a reference. RiOS has patented this reference. It includes a hierarchical structure with a single reference that can represent many segments and therefore the ability to reduplicate a large amount of data. If the data has not been seen before, the segments are compressed using Lempel-Ziv based algorithm and sent across to the RiOS appliance on the other side of WAN. These segments are also stored on the counterpart appliance. The original data is reconstructed using the new data and references to the existing data and passed through to the client. LZ Compression allows peak compression ratios of 100:1 to be achieved.