Simple Powershell script to get file from FTP

Question

so I've been trying to figure out this script to try and get a daily file uploaded to our FTP server. I've used this script in the past with success since the files are uploaded with a consistent file name: IE: FILE_NAME_2016-05-25.csv (Where the current days date is always formatted like this)

So this works when the file name maintains this consistency, but I need to use it to get a file that has an inconsistent value, appended to the end. IE: FILE_NAME_2016-05-25_0503 (The last value never being consistent).

So is there a way to use a wildcard character at the end of the file/folder variable to account for this?

My code (Note I am very new to powershell)

#Get's file from the FTP using today's date
Write-Host "Please wait while your file downloads"
$server = "ftp.server.net" 
$user = "user"
$pass = "pass"
$invocation = (Get-Variable MyInvocation).Value
$localpath = Split-Path $invocation.MyCommand.Path
$TodayDate = (get-date)
$FileNameDate = date $TodayDate -f yyyy-MM-dd
$remotefilepath = "inbox/sub1/sub2/FILE_NAME_$FileNameDate_[*].txt"
$localfilename = "FILE_NAME_$FileNameDate.csv"
$localfilelocation = "$localpath\$localfilename"
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user, $pass)
$uri = New-Object System.Uri(“ftp://$server/$remotefilepath”)
$webclient.DownloadFile($uri, $localfilelocation)

I tried to use a wildcard character at the end but it appears I am doing something wrong

I'm voting to close this question as off-topic because it's not about programming; it's about how to find files by wildcard with FTP commands. Try Super User? — briantist
– briantist, Commented May 25, 2016 at 18:58
I'm using this script to automate the processing and downloading of a file from an FTP to import into a SQL server. Normally we would use SSIS to do this, but because we don't have a license for MSSQL we have to go about it different ways. My apologies, but I thought that powershell scripts and automation tasks were part of the programming umbrella for stackoverflow — mattlore
– mattlore, Commented May 25, 2016 at 19:02
My point was that your script seems to work fine for sending FTP commands, you just don't know which commands to send. I'm still not certain it's on-topic, but I'm not certain it's off-topic either so I've retracted my close vote and I'll let others decide. — briantist
– briantist, Commented May 25, 2016 at 19:07

Oggew · Accepted Answer · 2020-12-15 06:08:33Z

1

Hi please try this.

Write-Host "Please wait while your file downloads"

#Function to get all files
function Get-FtpDir ($url,$credentials)
{
    $request = [Net.WebRequest]::Create($url)
    $request.Credentials = $credentials
    $request.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
    $response = $request.GetResponse()
    $reader = New-Object IO.StreamReader $response.GetResponseStream()
    $readline = $reader.ReadLine()
    $output = New-Object System.Collections.Generic.List[System.Object]
    while ($readline -ne $null)
    {
        $output.Add($readline)
        $readline = $reader.ReadLine()
    }
    $reader.Close()
    $response.Close()
    $output
}

$server = "ftp.server.net"
$user = "user"
$pass = "pass"
$invocation = (Get-Variable MyInvocation).Value
$localpath = Split-Path $invocation.MyCommand.Path
$TodayDate = (get-date)
$FileNameDate = date $TodayDate -f yyyy-MM-dd
$remotefilepath = "inbox/sub1/sub2"
$localfilename = "FILE_NAME_$FileNameDate.csv"
$localfilelocation = "$localpath\$localfilename"
$uri = New-Object System.Uri(“ftp://$server/$remotefilepath”)

#List of all files on FTP-Server
$files = Get-FTPDir $uri -credentials (New-Object System.Net.NetworkCredential($user, $pass))

foreach ($file in $files)
{
    if ($file -like "FILE_NAME_$($FileNameDate)_*.txt")
    {
        $file
        $fileuri = New-Object System.Uri(“ftp://$server/$remotefilepath/$file”)
        $webclient = New-Object System.Net.WebClient
        $webclient.Credentials = New-Object System.Net.NetworkCredential($user, $pass)
        $webclient.DownloadFile($fileuri, $localfilelocation)
    }
}

edited Dec 15, 2020 at 6:08

answered May 26, 2016 at 14:12

Oggew

3661 silver badge11 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

mattlore Over a year ago

Awesome! This works like a charm!! I also feel somewhat accomplished that my own research was leading me down to the IO.Streamreader and read line method to pull the data from a stream. This whole thing has been a rather interesting learning experience, but thank you!

Anshul Choure Over a year ago

Hi.. for me its failing and saying remote server requires ssl. can you please help me in estabilishing connection with https

LeeM Over a year ago

if anyone runs into the "protocol violation" error in .NET 4.5, read this article and try the first suggestion. The rest of the article explains the issue. :bhargavs.com/index.php/2014/03/17/…

Bacon Bits · Accepted Answer · 2016-05-25 20:23:57Z

I would look at using WinSCP's .Net library, which provides a more complete set of FTP commands. Or you could use FtpWebRequest, which also provides the functionality you're looking for since you can enumerate the remote files and then get the ones you want, but might be more cumbersome to use since all data transfers are streams. C# code is pretty portable to PowerShell, but it might be a bit steep for someone new so you might want to stick to WinSCP's .Net library.

Another alternative is to build a script file and use ftp.exe to get it. Here is an old doc that tells you what to do. You'll want to use the mget command instead. One drawback is that you've got to write the password to a text file, but, well, you technically already do that anyways since it's in your PowerShell script. You could do something like:

Set-Content -Path $FtpScriptFile -Encoding ASCII -Value "open ""ftp://$server/$remotefilepath"""
Add-Content -Path $FtpScriptFile -Encoding ASCII -Value "$user"
Add-Content -Path $FtpScriptFile -Encoding ASCII -Value "$pass"
Add-Content -Path $FtpScriptFile -Encoding ASCII -Value "cd ""inbox/sub1/sub2/"""
Add-Content -Path $FtpScriptFile -Encoding ASCII -Value "lcd ""$localpath"""
Add-Content -Path $FtpScriptFile -Encoding ASCII -Value "bin"
Add-Content -Path $FtpScriptFile -Encoding ASCII -Value "mget ""FILE_NAME_$FileNameDate_[*].txt"""
Add-Content -Path $FtpScriptFile -Encoding ASCII -Value "bye"
C:\Windows\System32\ftp.exe -s:"$FtpScriptFile"

(Note: I have not tested that at all, and it's not necessarily the best way to do this.)

Additionally, a lot of your script could be optimized. You're doing some backflips to get some simple values. (Get-Variable MyInvocation).Value is the same thing as $MyInvocation. The only reason you might need Get-Variable is to list assigned variables and to use dynamic variables -- that is, variables whose names you don't know until runtime... variable variable names.

This:

$invocation = (Get-Variable MyInvocation).Value
$localpath = Split-Path $invocation.MyCommand.Path
$TodayDate = (get-date)
$FileNameDate = date $TodayDate -f yyyy-MM-dd

Can just be:

$LocalPath = Split-Path $MyInvocation.MyCommand.Path
$FileNameDate = Get-Date -Format 'yyyy-MM-dd'

I also wonder if this is what's causing you problems:

$remotefilepath = "inbox/sub1/sub2/FILE_NAME_$FileNameDate_[*].txt"

When I have that in PowerShell ISE, it looks like the system thinks the variable is $FileNameDate_. Try:

$remotefilepath = "inbox/sub1/sub2/FILE_NAME_$($FileNameDate)_[*].txt"

That's how most people write it.

Personally, I'd probably write your script like so:

$RemoteFilePath = 'inbox/sub1/sub2/FILE_NAME_{0}_[*].txt' -f $FileNameDate
$LocalFileName = 'FILE_NAME_{0}.csv' -f $FileNameDate
$LocalFileLocation = Join-Path -Path $LocalPath -ChildPath $LocalFileName

That's using the format operator (see Get-Help about_Operators -ShowWindow) and the Join-Path cmdlet. I find it easier to read.

Awesome thanks for all the advice. I've used your advice to at least trim down and make my script more efficient. Unfortunately, it appears that even though your advice fixed the system variable issues with the file name, it's still outputting FILE_NAME_2016-05-25_[*] with a "File not found" error. So I'll dig around in those links you gave me and try and find another way to do this, since I don't think FTP requests handle the wildcards like local files do in powershell
@mattlore No, they do. That's what the mget and mput commands are for. Run ftp.exe interactively. you might need to use the command mget FILE_NAME_2016-05-25_*.*.

Collectives™ on Stack Overflow

Simple Powershell script to get file from FTP

2 Answers 2

3 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related