URGENT...Averaging out data anomalies

[danifj]

My name is Danielle Jacob and I am a Masters (through) Research student at Bangor University, North Wales.

[email removed]

I have been using Python now for about 3 months (but only once a fortnight for a few hours at a time...so I am still very new)

My supervisor extracted some tidal range data for a location in Wales (Holyhead) for 2011. For those of you who know tidal ranges it is supposed to look like a series of double helix oscillations. When I got the data there were some anomalies, such as a result of +40 (normally results should not exceed 5). These look like peaks in Python graphs.

There are a series of anomalies which I am able to identify. I asked my supervisor and he simply said to look up the code for creating an average of the before and after points to 'fake' the data.

What I want to know please is how to do this.

1. Identify the anomalies.
2. Take the before and after points and average them.

I have a graph set up.... I just need to change the appearance of the data on the graph. I can send you some of the code and a screen grab of the current graph.

Thank you for your time

Dani

[danifj]

This is my script - to be honest I hardly understand any of it.

import numpy as np
import pylab 
import string
import datetime as dt
import matplotlib.ticker as ticker
import matplotlib.dates as mdates

file_hh = open("C:/Users/Dani/Documents/Danielle/MRes/Tide Tables/Holyhead/2011HOL.txt",'r')
hh_data = file_hh.readlines()
file_hh.close()
n_data_hh=len(hh_data)

#ASLVBG02 = sealevel

hh_date = []
hh_sealevel = []

# Loop over every line in the dataset
for yind in range(11,n_data_hh):
#   | Indent starts here

    line_split = string.split(hh_data[yind])

    # Check we aren't looking at a blank line at the end
    if len(line_split) > 1:
    # | Indent starts here
      #print line_split

      date_split = string.split(line_split[1],'/')   
      time_split = string.split(line_split[2],':')

      if line_split[3][-1] == "M" or line_split[3][-1] == "N" or line_split[3][-1] == "T":
        tide = float(line_split[3][:-1])
      else:
        tide = float(line_split[3])
      #print date_split,time_split,tide

      year = int(date_split[0])
      month = int(date_split[1])
      day = int(date_split[2])
      hr = int(time_split[0])
      mn = int(time_split[1])

      if tide > -98.000:
        hh_date.append(dt.datetime(year,month,day,hr,mn,0))
        hh_sealevel.append(tide)

fig1 = pylab.figure(figsize=(8,6))

#figure size changes the size of the graph

ax1 = fig1.add_axes([0.1,0.1,0.8,0.8])
ax1.plot(hh_date,hh_sealevel)
ax1.set_xlabel('Time')
ax1.set_ylabel('Tidal Ranges')
ax1.set_title('Holyhead Tides')
#fig1.savefig('C:\Users\Dani\Desktop\Windconditions.png',dpi=300)
#dpi is dots per inch 300 is reasonable
#print shows this on the screen
pylab.show()

[Xantipius]

Dani,
email to me ( [email removed] ) the data file.
And I will try to understand what exactly have to be done.

Vitali

[TrustyTony]

low, high  = 10, 100

values = [12, 14, -34, 25, 200, 23, -23, 45, 23]

for i in range(1,len(values)-1):
    if values[i] < low or values[i] > high:
        # assuming that first value and last value are not off
        # and neighbouring values are not off
        values[i] = (values[i-1] + values[i+1]) / 2.0
print values

[Xantipius]

I think pyTony's code is quite enough a solution.