I don't remember where but i found this download function (geturl) that works great if i just send one url to the function.
But as you can see i want to execute geturl() for each line in url.txt witch contains url:s.
It still downloads but instead of a progressbar i just get newlines printed with "base" on it.
I don't get it, should i read the file in some other way?
def download(): for line in open('url.txt', 'r').readlines(): url = line base = url[url.rindex('/')+1:] geturl(url, base) def _reporthook(numblocks, blocksize, filesize, url=None): base = os.path.basename(url) try: percent = min((numblocks*blocksize*100)/filesize, 100) except: percent = 100 if numblocks != 0: sys.stdout.write("\b"*70) sys.stdout.write("%-66s%3d%%" % (base, percent)) def geturl(url, dst): #print "get url '%s' to '%s'" % (url, dst) if sys.stdout.isatty(): urllib.urlretrieve(url, dst, lambda nb, bs, fs, url=url: _reporthook(nb,bs,fs,url)) sys.stdout.write('\n') else: urllib.urlretrieve(url, dst) os.system("clear")
The files i try to download is about 100mb and it often fails with:
urllib.ContentTooShortError: retrieval incomplete: got only 36438016 out of 112266771 bytes
Is there a better way to download files with python?