Two years ago I began working on SmartTorrent, which was a sentiment analysis based torrent search engine. The goal of the project was to rank torrent searches based on user content such as comments and determine whether or not a torrent was safe to download. At the time this seemed like a feasible goal, however as the project grew I began to realize how incredibly complex the problem actually was. I realized my approach was inherently flawed as I was assuming some level of consistency in semantic structure of the content I was analyzing. Frustrated by this and the monolithic pile of crap the tool had become I decided to discontinue work on the project and begin work on a more feasible one; automating various aspects of network analysis for incident responders.
Actually, this has taken the form of several projects the two foremost being a web-based (I hate the word “cloud”) packet-capture (PCAP) analysis engine (think VirusTotal with PCAPs) and a NetFlow log visualizer which identifies top-talkers, potential lateral movement, and other incident response related metrics.
The web-based PCAP analysis engine allows a user to upload a PCAP file to our site, where it is offloaded to a processing node, replayed over a virtual network interface, and analyzed by several IDSs. The resulting analysis will return:
- Protocols found within the packet.
- Detailed logs of all connections
- Signatures fired.
- A list of related PCAP submissions containing similar data.
The obvious value of this tool comes from it’s ability to group similar packet-captures into one consolidated view. This allows an analyst to search our database using indicators such as IP, host-name, URLs, etc. and receive results which could be used to extend existing blacklists.
Over the next few months I will go into greater detail about each of these projects as I add features.
The link to the Netflow log visualizer can be found here. Please feel free to fork and improve.