Saturday, September 4, 2010

Data Visualization

One of the most useful ways to communicate or understand information is to visualize the data. In security this is vital due to the sheer volumes of data you often have to contend with i.e. from the secviz community.

Splunk can enable you to quickly find and parse data, however you often need to send the data to something like graphviz so that I can see it.

I've written an app that adds a new search command to Splunk that will enable you to generate a graph from your search data.


* | viz field1=ip1 field2=ip2 label=proto flatten=true file=/tmp/network1.png
 * | viz field1=ip1 field2=ip2 label=proto flatten=true file=/tmp/network2.png rankdir=RL


One of the nice features is that you can add graphviz options directly to the command i.e. rankdir=RL

The app can be obtained via http://github.com/marinus, comments & patches are welcome.



2 comments:

  1. There is also the afterglow application for Splunk. You can simply type in the search and it will show the resulting graph in the search result area. No addition of commands needed. The thing I am currently working on is afterglow cloud. It allows for configuration of the graphs directly in the userinterface with drop downs and such. Check it out at http://afterglow.sf.net

    ReplyDelete
  2. Raffy I'll have a look. Viz is for arbitrary relationships. I'm thinking of extending it to do multiple dataset relationships i.e. I could graph cell phone calling records fields[msisdn, msidn_to, status] and country fields[msisdn, location].

    viz fields=msisdn, msisdn_to, status; msisdn, location, ""





    i.e. viz fields=x,y,z;x,a,b;etc where x,y,z are perhaps

    ReplyDelete