C# Directory Browser / Indexing

[AleMonteiro]

Good night dani people.

I'm current developing a project that needs an web API to list all directories and files, and also provide a method to search by name.

I already got it working with Sytem.IO.DirectoryInfo at real time.

//Folders
foreach (DirectoryInfo info in _dirInfo.GetDirectories())
{
}

//and files
IEnumerable<FileInfo> filesInfo = _dirInfo.GetFilesByExtensions(extensions);

The aim is to provide a fast and smart(thumbs and media type) web service to browse the contents of specified media folders.

Worried about performace with multiple clients listing large folders I'd like to ask suggestions for better solutions.
Is indexing in a file or database worth? Or is DirectoryInfo the way to go?

Thanks all!

[AleMonteiro]

Meanwhile I found this DbfDotNet. Any toughts?

[DaveAmour]

I haven't done this for a long time but used to use something called Index Server. This indexed certain folders which you specified and you queried index server much like you query a database - this was mainly for searching file contents I think.

You can certainly see what kind of simillar technologies are available these days. Depends if you are searching file names/types only or contents as well.

Also how often do your folder contents change? If not too often then caching is a defination consideration.

Also are you searching just in file names or contents as well?

If just the former you could cache the whole thing in memory and even cache search results in memory too so if person a searches the same as person b did 10 minutes ago then the results are there in memory already.

Hope that helps.

[DaveAmour]

Another easy performance improvement with your current code - use a paralyzed foreach. These can add a lot of performance very easily.

[AleMonteiro]

Thanks for the replay, I'll take a look at Index Servers.

The search at first will be only by name and file type. I'd like to add tags too, but it's not a pre-req.

About numbers I'd say 50k initial files with 100+ modifications(delete/insert), per day, accross different folders.