Benford's law provides a useful heuristic to detect data that has been produced by a person. This is very useful to detect fraud, tampering, vote rigging and other activities where one needs a little help. It appears thought that the application of Benford's law is more of an art than a science and rather than being the smoking gun one would like, it serves as the starting point for an investigation or a trigger for caution.

I've developed a Splunk App that adds a new command to the Splunk search language that calculates the first digit distribution, which can then be used to graph the field of interest.

** | benford field=price | table digit price benford*

Other digits can be selected as follows

** | benford field=price digit=2 | table digit price benford*

Here's some sample transactions I generated

The benford command will calculate the distribution of the first digit and produce a table, which can be graphed.

The following graph illustrates the digit distribution compared to the benford distribution.

The following graph was created using real transactional data.

You can download a copy of the App from http://bit.ly/9JBoPm

ReplyDeleteEven more interesting

ReplyDelete"Benford's Law And A Theory of Everything"

A new relationship between Benford's Law and the statistics of fundamental physics may hint at a deeper theory of everything.

http://pubsub.com/Benfords-Law-And-A-Theory-of-Everything_Tech-Physics-4DWoeoTi3oA,7nezUbfaxqEE