We're a community of 1.1M IT Pros here for help, advice, solutions, professional growth and fun. Join us!
1,080,644 Members — Technology Publication meets Social Media
Username:
Password:
Lost login information?
Start New Discussion Reply to this Discussion

Statistics::Descriptive gives deviant Standard Deviation?

I know next to nothing about statistics so tried Statistics::Basic because it seemed easy to use. Then I tried Statistics::Descriptive because it has functions that Statistics::Basic lacks. What I did not expect was to get a different result when calculating Standard Deviation using Statistics::Descriptive than when using the other module or a subroutine copied form Yahoo Answers. Am I doing something wrong, or does Statistics::Descriptive have a deviant way of calculating deviations?

#!/usr/bin/perl
use strict;
use warnings;

use Statistics::Basic qw(:all);
use Statistics::Descriptive;

my @d = (5,10,5,100,150);
print 'StdDev according to Basic is ', stddev(@d), "\n"; #Basic

my $stat = Statistics::Descriptive::Full->new();
$stat->add_data(@d);
print 'StdDev according to Descriptive is ', $stat->standard_deviation(), "\n"; #Descriptive

print 'StdDev according to subroutine is ', standard_deviation(@d) . "\n";

sub standard_deviation {
    my (@numbers) = @_;

    #Prevent division by 0 error in case you get junk data
    return undef unless ( scalar(@numbers) );

    # Step 1, find the mean of the numbers
    my $total1 = 0;
    foreach my $num (@numbers) {
        $total1 += $num;
    }
    my $mean1 = $total1 / ( scalar @numbers );

    # Step 2, find the mean of the squares of the differences
    # between each number and the mean
    my $total2 = 0;
    foreach my $num (@numbers) {
        $total2 += ( $mean1 - $num )**2;
    }
    my $mean2 = $total2 / ( scalar @numbers );

    # Step 3, standard deviation is the square root of the
    # above mean
    my $std_dev = sqrt($mean2);
    return $std_dev;
}

Gives the following output:

StdDev according to Basic is 60.12
StdDev according to Descriptive is 67.2123500556259
StdDev according to subroutine is 60.1165534607565
1
Contributor
1
Reply
18 Hours
Discussion Span
2 Years Ago
Last Updated
2
Views
Question
Answered
d5e5
Practically a Posting Shark
831 posts since Sep 2009
Reputation Points: 162
Solved Threads: 163
Skill Endorsements: 1

After further googling I found a post titled Perl Standard Deviation function is wrong that explains that there are at least two ways of calculating standard deviation which give noticeably different results for small data lists such as my examples use.

Conclusion: the values for standard deviation calculated by Statistics::Basic and Statistics::Descriptive differ for small data sets but this doesn't mean either value is wrong. What statistics module you use for calculating standard deviation depends on what calculation method you, your colleagues and your boss agree on.

d5e5
Practically a Posting Shark
831 posts since Sep 2009
Reputation Points: 162
Solved Threads: 163
Skill Endorsements: 1
Question Self-Answered as of 2 Years Ago

This question has already been solved: Start a new discussion instead

Post: Markdown Syntax: Formatting Help
 
You
View similar articles that have also been tagged:
 
© 2013 DaniWeb® LLC
Page generated in 0.0641 seconds using 2.68MB