• Download FTPDownloadTool - 15.12 KB

Introduction

I have a Windows 2008 server rented from a hosting company that is running several SQL Server 2005 databases, among other things. Each morning at 3:00AM, a program I wrote will shutdown the database server, copy database files to a backup folder (the files will be renamed by adding a date string), and then restart the database server. This way I can save a copy of data for each day, just in case a server crash will corrupt my data. Since there is limited storage on the Windows 2008 server, my program also deletes files in the backup folder that are more than 7 days old.

What if a more serious disaster happens? Say, the hard disk on the server gets toasted? What if I need to look at data snapshot that is more than 7 days old?  What I need is a tool that download files from the backup folder on the remote server to my home machine via FTP.

The FTP Tool

My ftp tool is a .NET console application, it does one thing and one thing only. That is, downloading contents of a remote folder to a local folder. It can be scheduled to run once a day. Here is how it works:

1. Read remote folder path and local folder path from configuration file.
2. Get list of files and subfolders of the remote folder.
3. For each file, download it from the remote folder to the local folder.
4. For each subfolder, create a subfolder with the same name on the local folder and go to step 2 (recursively download files/folders of the remote subfolder).

Here is a sample of configuration file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key ="FTPUserName" value ="MyUserName"/>
    <add key ="FTPPassword" value ="MyPassword"/>
    <add key ="FTPURL" value ="ftp://MyServer.com"/>
    <add key ="FTPRemoteFolder" value ="%2fMyRemoteFolder"/>
    <add key ="FTPLocalFolder" value ="C:\MyLocalFolder"/>
    <add key ="CleanupDays" value ="0"/>
    <add key ="TraceDirectory" value ="c:\logs\ftp"/>
    <add key ="TraceFileName" value ="FTPDownloadTool"/>
    <add key ="TraceLevel" value ="4"/>
  </appSettings>
</configuration> 

The value of FTPRemoteFolder specifies the path of the remote folder on the server. String %2f indicates the path is relative to the FTP root directory. Without %2f the path will be relative to the ftp user's home directory. 

If you don't want to keep all files downloaded to your local folder, you can specify a positive value for CleanupDays, say 60, which will cause all files in your local folder that are more than 60 days old to be deleted automatically!  

Note: The program is designed to run in the background without human intervention. Any error will be written to the log file defined in the configuration file. In case of error, such as loss of network connection, the ftp commands will be retried until all files and subfolders in the remote folder have been downloaded successfully.

The Code

Here is the main method. As you can see, it just calls the DownloadFolder method which does the main work. In case of error, the code will pause (sleep) for 60 seconds then call DownloadFolder again.

static void Main(string[] args)
{
    Tools.Trace.WriteTrace(4, "FTPDownloadTool starting ...");
    bool bDoneForToday = false;
    while (bDoneForToday == false)
    {
        try
        {
            DownloadFolder(FTPRemoteFolder, FTPLocalFolder);
            bDoneForToday = true;   
        }
        catch (Exception bug1)
        {
            Tools.Trace.WriteTrace(bug1);
            System.Threading.Thread.Sleep(IdlePause);
        }
    }
    Tools.Trace.WriteTrace(4, "... FTPDownloadTool ended");
    System.Diagnostics.Process.GetCurrentProcess().Kill();
}

The DownloadFolder method uses .NET FtpWebRequest class to get file list from remote server and to download files.  Here is the code to get file list:

FtpWebRequest req = (FtpWebRequest)FtpWebRequest.Create(FTPURL + "/" + RemoteFolder);
req.Credentials = new NetworkCredential(FTPUserName, FTPPassword);
req.EnableSsl = false;
req.UsePassive = false;
req.UseBinary = true;
req.KeepAlive = false;
req.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream());
string[] pRemoteFiles = reader.ReadToEnd().Split("\n".ToCharArray());
reader.Close();

For each subfolder, the DownloadFolder method will be called recursively. For each file that has not been downloaded already, DownloadFolder will download the file to the local folder. Here is code to download a file:

Tools.Trace.WriteTrace(4, "Downloading file: ", sRemoteFile, ", file size: ", fsize, ", file date/time: ", ftime.ToString("MM/dd/yyyy HH:mm:ss"));
req = (FtpWebRequest)FtpWebRequest.Create(FTPURL + "/" + RemoteFolder + "/" + sRemoteFile);
req.Credentials = new NetworkCredential(FTPUserName, FTPPassword);
req.EnableSsl = false;
req.UsePassive = false;
req.UseBinary = true;
req.KeepAlive = false;
req.Method = WebRequestMethods.Ftp.DownloadFile;
BinaryReader breader = new BinaryReader(req.GetResponse().GetResponseStream());
int nPos = 0;
while (true)
{
    int nSize = breader.Read(pDataBuffer, nPos, MaxFileSize - nPos);
    if (nSize <= 0) break;
    nPos += nSize;
    if (nPos >= MaxFileSize) break;
}
breader.Close();
if (nPos != fsize) throw new Exception("Wrong file size");
BinaryWriter bwriter = new BinaryWriter(new FileStream(LocalFolder + "\\" + sRemoteFile, FileMode.Create));
bwriter.Write(pDataBuffer, 0, nPos);
bwriter.Close();

Limitations

The program does not work with non-windows FTP servers. In fact I only tested it with Windows 2003 and Windows 2008 servers.

Also, it is not designed to run fast, everything is done in a single thread.

History

01/27/2011: Initial version posted. 

01/28/2011: Updated code. Modified article text (added code and explanations).