Directory Scanning (sort of)

[Matt Tacular]

I have been trying to search the web to find a way to code a python program that can scan a given directory (ex. "C:\Documents and Settings\Matt\Desktop") and assign every file name found there (ex. "test.py" and "essay.doc") to a seperate variable or inside a list.

Is there any way to do this? or something very similar?

Thanks all.

-Matt

[jrcagle]

If you want to do a recursive search, look up os.path.walk(). If you just want a shallow directory listing, try os.listdir().

Regards,
Jeff

[vegaseat]

The Python module glob will get you there ...

# get all the file types in a given directory:
import os, glob
# change to the desired directory
os.chdir("C:/Documents and Settings/Matt/Desktop")
# get all the files in that directory into a list
file_list = glob.glob("*.*")
# show the list
print file_list

If you want to list specific files use:

# for selected file types use ...
import os, glob
# change to the desired directory
os.chdir("D:/Python24/Atest/BS")
# get files with extension .py and .pyw in that directory into a list
file_list = glob.glob("*.py") + glob.glob("*.pyw")
# show the list
print file_list

[Matt Tacular]

Thanks a bunch guys, I can now do exactly what i asked for help with. :D

[Matt Tacular]

Is there a way though, to make it also scan sub directories?

[ghostdog74]

Is there a way though, to make it also scan sub directories?

Just an example only:

import os
allfiles = [] #store all files found
for root,dir,files in os.walk("C:/Documents and Settings/Matt/Desktop"):	
 	filelist = [ os.path.join(root,fi) for fi in files if fi.endswith(".doc") or fi.endswith(".txt") ]
 	for f in filelist: 
              allfiles.append(f)
	
	
for i in allfiles:
	print "file ", i

[derkin]

You want os.walk([path to dir here])

for d in os.walk("c:\"):    print d

[Matt Tacular]

Thanks alot guys :D

[Matt Tacular]

Is there actually anyway to make the post number 6 code example actually scan for every kind of file? i tried putting ".*" instead of ".txt" but it didnt work for me.

[vegaseat]

That example would not allow for wildcards, a talent of module glob only. However, you can simply leave out the if statement in the list comprehension line:

filelist = [ os.path.join(root,fi) for fi in files ]

[ghostdog74]

That example would not allow for wildcards, a talent of module glob only. However, you can simply leave out the if statement in the list comprehension line:

filelist = [ os.path.join(root,fi) for fi in files ]

thanks..yes i know.
I was giving an example on his first post.. doc and txt files. :)

[Ene Uran]

I enjoyed the flow of this discussion, learned a lot!

[Matt Tacular]

I tried putting in a network address "\\a219-4\c\" and I found that it did infact work. I got it to show me a list of all the files. But I was wondering if there was a way to get it to scan every networked computer attached to my computer, without having to type in a new address each time, and I know how to do that very thing myself. But I would need to get every address for the networked computers into a list, which is what I don't know how to do.

If anyone can help me with this or point me to a tutorial on how to do this, I would appreciate it alot.

Thanks.

[ghostdog74]

I assume you are working on win32 platform
if you have the win32 module, there is an example demo called win32netdemo.py and inside the source, there is a function call ServerEnum(). You can take a look at how to find network shares.
Another way is to call a system command that is able to find the network shares and put them into a file. Then from Python, open the file for processing..

[Matt Tacular]

I have another riddle sort of (to do with scanning directories). I want to modify the code so that it searches for a given file name (ex "test.py"). Once found, it uses that files parent directory, and searches every subdirectory in that directory, and puts those files in a seperate list. That way, if you always keep your files bunched together based on type (ex. a "python programming" folder) like I do, but if you happen to forget where it is, you can search for something like "test.py" and find all the files that are with it aswell.

I tried to find a way to do it, but failed horribly, any help is appreciated greatly! :D

-Matt

[vegaseat]

You might be able to modify the Python code snippet at:
http://www.daniweb.com/code/snippet527.html

[Matt Tacular]

So here is the program I have made so far using all the help from this thread so far:

# search for a file, and show all files found in that file's directory
 
import os
import pickle
 
def file_find(folder):
    """search for a filename fname starting in folder"""
    for root, dirs, files in os.walk(folder):
        for file in files:
            # make search case insensitive
            if fname.lower() == file.lower():
                return os.path.join(root)
    return None
 
# file to search for ...
fname = raw_input("File? ")
# folder to start search ...
folder = raw_input("Directory? ")
 
result = file_find(folder)
 
if result:
    print "File found -->", result
    print "\n===============================\n" 
    directory = result

    allfiles = [] #store all files found
    for root,dir,files in os.walk(directory):
            filelist = [ os.path.join(root,fi) for fi in files ] 
            for f in filelist: 
                  allfiles.append(f)
            
    fileAmount = len(allfiles)
            
    for i in allfiles:
            print "file ", i

    print "\n===============================\n"


    counter = 0
    while counter != fileAmount:

        showFile = allfiles[counter]

        counter += 1

        if counter == fileAmount:
            break
        
    raw_input("")
else:
    print "File %s not found" % fname

I am going to use this as a small part of a greater program for my grade 12 programming final project.

Thanks for all the help guys :D

[bumsfeld]

Make sure you test this out thoroughly! There are few things that might need to be changed.

[Matt Tacular]

What do you mean? it works well for me.

[vegaseat]

Just a few observations:

The function find_file() should have two arguments find_file(fname, folder). You are relying on fname to behave like a global variable, somewhat unsafe!

What are you using the module pickle for?

[Matt Tacular]

What are you using the module pickle for?

Well I updated my code to be alot more usefull, i was in the process of doing that when i posted, and i only posted the stable part at that time but forgot to remove pickle.

here is my current version of it:

import os
import random
import pickle

ques1 = raw_input("Scan dir [d] or find file [f]? ")

if ques1 == "d" or ques1 == "D":
    directory = raw_input("Directory: ")

    allfiles = [] #store all files found
    for root,dir,files in os.walk(directory):
            filelist = [ os.path.join(root,fi) for fi in files ] 
            for f in filelist: 
                  allfiles.append(f)
            
    for i in allfiles:
            print "file ", i

    print "\n===============================\n"

    file = open("files in dir.txt", "w")
    pickle.dump(allfiles, file)
    file.close()

if ques1 == "f" or ques1 == "F":
 
    def file_find(folder):
        """search for a filename fname starting in folder"""
        for root, dirs, files in os.walk(folder):
            for file in files:
                # make search case insensitive
                if fname.lower() == file.lower():
                    return os.path.join(root)
        return None

    def file_find2(folder, fname):
        """search for a filename fname starting in folder"""
        for root, dirs, files in os.walk(folder):
            for file in files:
                # make search case insensitive
                if fname.lower() == file.lower():
                    return os.path.join(root, fname)
        return None
     
    # file to search for ...
    fname = raw_input("File? ")

    # folder to start search ...
    folder = raw_input("Directory? ")
     
    result = file_find(folder)
    result2 = file_find2(folder, fname)
     
    if result:
        print "File found -->", result2
        print "\n===============================\nFollowing are the files in the same directory\n===============================\n" 
        directory = result

        allfiles = [] #store all files found
        for root,dir,files in os.walk(directory):
                filelist = [ os.path.join(root,fi) for fi in files ] 
                for f in filelist: 
                      allfiles.append(f)
                
        fileAmount = len(allfiles)
                
        for i in allfiles:
                print "file ", i

        print "\n===============================\n"
    
    file = open("files near file.txt", "w")
    pickle.dump(allfiles, file)
    file.close()

run = raw_input("Run a file [y]? ")

if run == "y" or run == "Y":

    if ques1 == "f" or ques1 == "F":
        ques2 = raw_input("Run file searched for? [y] ")

        if ques2 == "y" or ques2 == "Y":
            os.startfile(result2)

        else:
            ques3 = raw_input("Near file searched for? [y] ")

            if ques3 == "y" or ques3 == "Y":
                stringNum = input("Enter string number (from txt file, number ABOVE desired file): ")
                fileToRun = allfiles[stringNum]
                os.startfile(fileToRun)
                
            else:
                runFile = raw_input("File? (whole address) ")
                os.startfile(runFile)

    else:
        ques4 = raw_input("In List? [y] ")

        if ques4 == "y" or ques4 == "Y":
            stringNum = input("Enter string number (from txt file, number ABOVE desired file): ")
            fileToRun = allfiles[stringNum]
            os.startfile(fileToRun)
                
        else:
            runFile = raw_input("File? (whole address) ")
            os.startfile(runFile)

else:
    raw_input("Okay...")

I use pickle for the file saving because when i run the program without the idle it cuts off the top of the list, so i saved the list to file for viewing and found the list number is saved above the file name, so i used that to make it easy to run files or programs from the list.

I know there has to be a better way to do what i did with def file_find() and def file_find2()

I was doing that to get only the directory of the file, not the whole adress of the file. the second one is for the whole address.