Thursday, April 10, 2014

I will be exploring the Netmon option for Splunk in this post. Hopefully, I should be able to cover most of the technical details. This application is expected to capture http or https traffic only, hence it is much more similar to Fiddler than it is to Wireshark or NetMon since they are based on the lower layers of networking stack in the application. The principle in a packet capture tool such as this is to substitute the internet proxy with one from the application that has an address and port as say 127.0.0.1:8888. If we look at the System.Net.WebProxy class, we have the options to set the Address property and specify the web proxy for instances of the web request in an application. Global proxy settings are specified in the machine and application level configuration file. In Internet-Explorer when we make proxy settings per-machine rather than per user, we are forcing all users to use the same proxy settings rather than to use their own settings of the proxy. This works well for packet capture because when we switch the proxy we know that we will capture all traffic. In production environments, this is typically not an issue. Since our application sits between the WinInet and the corpnet proxy on these machines, it should be able to capture all the http/s traffic. We may have to call the WinInet methods by PInvoke or use the poshHTTP class as available on the Internet. WinInet exposes four methods InternetOpen, InternetCloseHandle, InternetSetOption, and InternetQueryOption to set and retrieve Internet settings. With the WebBrowser data structure, we can   push the settings to all applications accessing the internet.
Having talked about setting and removing our proxy, we now look at the other role of the proxy which is to play the man-in-the-middle. This is done via the forward option. Note that Splunk has an inbuilt option to read network traffic at a TCP port and localhost. The same can be used from the proxy to log packets. However the proxy we implement also needs to forward the packets outbound from the machine. And this is what we look at next.
Here we have to pay attention to three things:
First is that we should relay the packets hence we need to parse the destination and forward to that address and port (typically 80 or 443).
Second, we turn off the keep-alive and send the requests and responses as if new i.e. we use the connection-close on both sides.
Third, our forwarding is in both directions so we have the same implementation for source-destination basis.
With the above, we have barely scratched the surface but we know this is viable.
One important thing to mention here is that the the requests and responses will have parses be data because they have headers. By that we mean no matter what the source and destination is the requests and responses can be input to Splunk with a predetermined event schema. The value addition to regular packet capture is the Splunk search that we can now do to enable powerful analytics.

No comments:

Post a Comment