I read the message from the queen of the daniweb about surrounding the code with
code tags, but I did not understand the message at all.
I will enclose this code, and resend it, if necessary if some would explain the
"

" "

" tag dodad.

Basically i have a record with six fields; I have a function that reads from
stdin the range of record numbers, the field I want changed and the value.
I also have a function that lists the shelve and enums the indexes into
a readable info.
A third function just lists each "test" or object in the opened shelf file,
and enum- infos it into a readable form.

The problem is, if I call fill with a range of 1 to 48 or less, when listing, all is well.
If I try 1 to 49 or greater, some records get clobbered and the list function, which starts
at record one, dies on the first dead record, in most cases, "test1" the first record.

It is as if, the shelve wraps on itself at some size...

Note the list, starts 1 to N, and dies in the failure mode, but the entire record is
not squashed, because you can use the third function to read other objects, just not
the one that got clobbered:
Here is three files concatenated:(too bad you can't attach, or can you?)

usage:
you get prompts:
try 1,48
then
fmask,7

the next prompt try list
all works.

now try 1,50
fmask,7
list
the thing croaks on test1

class Interact:
	print "called interact"
	def __init__(self):
		self.prompt ='?'

	def run(self):
		#print "called interact run"
		while 1:
			command = self.readCommand()
			print "what command is it? %s" % command
			if not command:
				break
			result = self.evalCommand(command)
			if result:
				break
	def readCommand(self):
		#print "well, i am reading a command??"
		try:
			return raw_input(self.prompt)
		except:
			return None
class Record:
	def __init__(self, *args):
		for field, default in self.fields:
			#print " %s" % field
			if args:
				setattr(self, field, args[0])
				args = args[1:]
			else:
				print "init call fields %s" %self
				setattr(self, field, default)
	def save(testrecord):
                print "save call for record = %s" % testrecord.name
                name = getattr(testrecord, 'name')
                import shelve
                db = shelve.open('eride')
                db[name] = testrecord
                db.close()

	def basic(self):
		result = []
		for field, default in self.fields:
			result.append((field, getattr(self, field)))
		return result

	def extra(self):
		result = []
		for attr in self.__dict__.keys():
			for field, default in self.fields:
				if field == attr: break
			else:
				result.append((attr, getattr(self, attr)))
		return result

	def info(self): 
		return (self.basic(), self.extra())


class Test(Record):
        fields = [ ('name',''), ('fmask',0), ('bmask',0), ('cdesrc',0), ('rfsrc',0), ('sersrc',0), ('notes','na') ]


#!/usr/bin/python
import sys
import string
import shelve
from Int import Interact
from types import *
from test0 import Test
from enum import Enum

#choices are 26,27,52
bmask  = Enum('38400','57600','115200','230400')
fmask  = Enum('26','27','52')
cdesel = Enum('def','oscval','baudrate')

########################################
#GLOBALLY set test file
testfile='eride'

##set the test file here

class TestPak(Interact):
	def __init__(self):
		self.prompt = "tool ? [(f)illfield,(l)ist,(t)estcontents, (s)op] "
		#print "where did the prompt go?? %s " % self.prompt

	def evalCommand(self, name):
		#print "did we ever get to eval command?"
		if name[0] == 'f':
			self.fillfield()
		elif name[0] =='l':
			self.list()
		elif name[0] =='t':
			self.listtest()
		elif name[0]=='s':
		 	return 1	
		else:
			print "what? try again"

	def fillfield(self):
                db = shelve.open(testfile)
		print "input low,high indices of tests whose field you want to fill in"
		low,high = string.split(sys.stdin.readline(),',')
		lo = string.atoi(low)
		hi = string.atoi(high)
		field,val = string.split(raw_input("field,value?" ),',')
		if field =='name' or field =='notes':
			value = val
		else:
			value = string.atoi(val)
		print "input fieldname,value"
		for i in range(lo,(hi+1)):
			testname = "test%s" % i
			try:
			    object = db[testname]
			except:
			    #fill with default settings if no testname in file
			    object = Test()
			    object.__dict__['name']= "%s" % testname 

			object.__dict__[field] =value
			print "call save of test %s value %s" % (testname,value)
		 	object.save()
			print object.info()
		db.close()

	def list(self):
		import shelve
		db = shelve.open(testfile)
	        cnt = 1	
		while 1:
			try:
				name = getattr(db["%s" % ('test'+`cnt`)],'name')
			except:
				print "escaped on %s" % ('test'+`cnt`) 
				break
			print db[name].info()
			cnt = cnt +1
        def listtest(self):
                import shelve
                db = shelve.open(testfile)
                test = raw_input("test name? ")
                try:
                        object = db[test]
                except:
                        sys.stderr.write("no record of this name : %s in file : %s  "  % (test,testfile))

                print db[test].info()


if __name__== '__main__': TestPak().run()


[CODE\]

Editor's note:
for Python on Daniweb:
Please use the [code=python] and [/code] tag pair to enclose your python code.

general:
Please use the [code] and [/code] tag pair to enclose your code.

Okay since no one else want to respond the question, the answer is YES..
Apparently in redhat version 3.2, when you open a shelve you get a fixed
record of length 12288.
If you now preceed to fill in records of so many bytes per record, when the number of
records you keep adding, or appending as it were, eventually you get past 12288.
and then immediately the file size doubles to 24576. BUT, the record indexes now get all screwed up.

In the test code below is a section that calls for a new Test record, which will initialize the fields to default values, IFF there is no record in the file under the name. If the named file already exists, it just sets the attr to the new value.
If I fill in the field with a range of records from 1 to 25 or up to 1 to 84, the actually shelved record size stays at 12288. The first time you fill in the field in 1 to 84 records, you
will see the call to init. Repeat, and since the record aleady exists, the call is to the routine
that fills in the record.
BUT, if you try an fill in a field of 85 records (set range 1 to 85), the poop hits the fan.
When you list the file contents, what was the contents of test2 has been lost, and now
the record size instead of being 12288 is 24576. The file size doubled, but in doing so,
the record indexing gets screwed up.
??? Does this mean, if you use a shelve, you have to preallocate the largest size file
that you ever expect to use? If this is true, using a shelf as a place to store records is
only good if you always have the same number of records. This is kind of lame, isn't it?

code begins here (i add the [code] from sites request, but the indenting doesn't kick in.
I do no know how this site adds the green spyglass "code" dodad)

#!/usr/bin/python
import sys
import string
import shelve
from Int import Interact
from types import *
from enum import Enum

#choices are 26,27,52
bmask  = Enum('38400','57600','115200','230400')
fmask  = Enum('26','27','52')
cdesel = Enum('def','oscval','baudrate')

########################################
#GLOBALLY set test file
testfile='testfile'

class Record:
    def __init__(self, *args):
        print "TRACE0:init call"
        for field, default in self.fields:
            #print " %s" % field
            if args:
                setattr(self, field, args[0])
                args = args[1:]
            else:
                print "init call fields %s" %self
                setattr(self, field, default)
    def save(testrecord, testfile):
                print "TRACE1:save call for record = %s" % testrecord.name
                name = getattr(testrecord, 'name')
                import shelve
                db = shelve.open(testfile)
                db[name] = testrecord

    def basic(self):
        result = []
        for field, default in self.fields:
            result.append((field, getattr(self, field)))
        return result

    def info(self): 
        return self.basic()
class Test(Record):
        fields = [ ('name',''), ('fmask',0), ('bmask',0)]


class TestPak(Interact):
    def __init__(self):
        self.prompt = "tool ? [(f)illfield,(l)ist,(s)op] "
        #print "where did the prompt go?? %s " % self.prompt

    def evalCommand(self, name):
        #print "did we ever get to eval command?"
        if name[0] == 'f':
            self.fillfield()
        elif name[0] =='l':
            self.listtest()
        elif name[0]=='s':
            return 1    
        else:
            print "what? try again"

    def fillfield(self):
                db = shelve.open(testfile)
        print "input low,high indices of tests whose field you want to fill in"
        low,high = string.split(sys.stdin.readline(),',')
        lo = string.atoi(low)
        hi = string.atoi(high)
        print"OKAY shithead, what are the values: %d %d" % (lo, hi)
        field,val = string.split(raw_input("field,value?" ),',')
        if field =='name':
            value = val
        else:
            value = string.atoi(val)
        print "TRACE FILL: %s %d" % (field,value)
        for i in range(lo,(hi+1)):
            testname = "test%s" % i
            print "TRACE: test name is %s" % testname
            try:
                object = db[testname]
            except:
                #fill with default settings if no testname in file
                print "TRACEtry2: init, object does not exist"
                object = Test()
                object.__dict__['name']= "%s" % testname 

            object.__dict__[field] =value
            print "TRACE: fill2:call save of test %s value %d" % (testname,value)
            object.save(testfile)
            print object.info()
        db.close()

    def listtest(self):
        import shelve
        db = shelve.open(testfile)
            cnt = 1 
        while 1:
            try:
                name = getattr(db["%s" % ('test'+`cnt`)],'name')
            except:
                print "escaped on %s" % ('test'+`cnt`) 
                break
            print db[name].info()
            cnt = cnt +1


if __name__== '__main__': TestPak().run()

Edited 3 Years Ago by diafol: fixed formatting

I found the error, thanks to the kickbutt moderator.
I tried to break his shelve code the same way mine broke, but could not.
Went back and compared the two, and my error was that
I open the shelve in a routine to look at the key, to know whether to modify a current record or open a new one. Then, I call a routine to save the record, in in this save routine
I RE-open the shelve.. THIS is what messes stuff up.
When i move the save logic into the same define as the one the sets up the modified-new record, and then just save the record without re opening, my phantom index screw up
went away. Thanks to the moderator for his example. It was a little work, but I found it.

This article has been dead for over six months. Start a new discussion instead.