Member Avatar for maf_9000

i am building a simple sync program,
whan the program starts it takes all the bandwith,
i need to set the speed limit,
i am using using ftplib

file = open(local_file_path, 'r')
    print ftp.storlines('STOR '+ file_name , file)
  • 3 Contributors
  • 3 Replies
  • 1K Views
  • 4 Days Discussion Span
  • Latest Post Latest Post by Gribouillis

Dani AI

Generated

Short practical summary: ' idea (a file-like object that delays between reads) is on the right track for controlling upload rate. is correct to point out the call-site mistake: ftp.storlines() wants a file-like object (with readline()), not the result of calling readline() once. For more precise, byte-level throttling use storbinary() and supply a file-like object that implements read(size) and enforces a bytes-per-second target.

Example throttle implementation (Python 3 style):

import time

class ThrottledFile:
    def __init__(self, path, bytes_per_sec, blocksize=8192):
        self.fp = open(path, 'rb')
        self.bps = float(bytes_per_sec)
        self.blocksize = blocksize
        self.start = time.monotonic()
        self.sent = 0

    def read(self, size=None):
        size = size or self.blocksize
        data = self.fp.read(size)
        if not data:
            return b''
        self.sent += len(data)
        elapsed = time.monotonic() - self.start
        expected = self.sent / self.bps
        if expected > elapsed:
            time.sleep(expected - elapsed)
        return data

    def close(self):
        self.fp.close()

Use it with ftp.storbinary('STOR '+name, tf, blocksize=tf.blocksize) and call tf.close() afterward. Pick blocksize and bytes_per_sec so blocksize / bytes_per_sec gives a reasonable delay (smaller blocks = smoother control). Note this is application-level throttling — TCP and OS buffers can add variance. For strict limits use OS traffic shaping (tc on Linux, platform tools on Windows). If uploading text and you must use storlines(), the same idea applies but implement readline() instead of read().

Recommended Answers

All 3 Replies

Member Avatar for Gribouillis

You could use a modified file object which sleeps between the lines. The following code works for me

# python 2
HOST, USER, PASSWD = "", "", "" # <--- your values here

from ftplib import FTP
from time import sleep

class SlowFile(object):
    def __init__(self, name, mode="r", delay = 0.1):
        self.src = open(name, mode)
        self.delay = delay
        
    def readline(self):
        line = self.src.readline()
        if line:
            sleep(self.delay) # sleep between the lines
        return line

def main():
    ftp = FTP(HOST, USER, PASSWD)
    try:
        name = "foo.txt"
        src = SlowFile(name, "r", delay = 0.2)
        ftp.storlines("STOR " + name, src)
    finally:
        ftp.quit()
    
if __name__ == "__main__":
    main()

This should be very slow. You may set the delay according to your file's size.

Edited by Gribouillis because: n/a
Member Avatar for richieking

Grib. SlowFile.readline was not called therefore the delay will not work.
it should be like......

def main():
    ftp = FTP(HOST, USER, PASSWD)
    try:
        name = "foo.txt"
        src = SlowFile(name, "r", delay = 0.2)
        ftp.storlines("STOR " + name, src.readline()) # this part....
    finally:
        ftp.quit()
    
if __name__ == "__main__":
    main()
Edited by richieking because: n/a
Member Avatar for Gribouillis

Grib. SlowFile.readline was not called therefore the delay will not work.
it should be like......

def main():
    ftp = FTP(HOST, USER, PASSWD)
    try:
        name = "foo.txt"
        src = SlowFile(name, "r", delay = 0.2)
        ftp.storlines("STOR " + name, src.readline()) # this part....
    finally:
        ftp.quit()
    
if __name__ == "__main__":
    main()

I don't think so: first, the code worked for me as it is, second, the second argument of ftp.storlines() is an open file object with a readline() method, according to the python documentation. In your snippet, you're passing a string (the first line of the file).

Edited by Gribouillis because: n/a
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.