Start-AzStorageBlobCopy in powersell

Question

Using powershell, I am trying to get the azure webapp log files and copy them to azure storage account container and then delete

$webAppName  = "WA01"
$resourceGroupName = "RG01"
$storageAccountName = "SA01"
$containerName = "container01"
$numberOfDays = 90


# Generate Kudu API URL for log files
$kuduApiUrl = "https://${webAppName}.scm.azurewebsites.net/api/vfs/LogFiles/"

# Retrieve publishing profile for the Azure Web App
Write-Host "Retrieving publishing profile for $webAppName..."
$publishingProfile = Get-AzWebAppPublishingProfile -ResourceGroupName $resourceGroupName -Name $webAppName

# Parse publishing profile XML to get credentials
[xml]$publishingProfileXml = $publishingProfile
$publishingUserName = $publishingProfileXml.publishData.publishProfile[0].userName
$publishingPassword = $publishingProfileXml.publishData.publishProfile[0].userPWD

# Convert credentials to base64
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes("${publishingUserName}:${publishingPassword}"))
Write-Host "Retrieving log files from $webAppName..."
    $logFilesToBackup = (Invoke-RestMethod -Uri $kuduApiUrl -Headers @{Authorization=("Basic $base64AuthInfo")} -Method GET).Where({ 
            $mtimeValue = $_.mtime.Split('.')[0]  # Remove milliseconds
            $mtimeValue = $mtimeValue.Substring(0, 19)  # Remove time zone offset
            [DateTime]::ParseExact($mtimeValue, "yyyy-MM-dd'T'HH:mm:ss", [System.Globalization.CultureInfo]::InvariantCulture) -lt      $cutoffDate 
        })

    # Perform backup actions...
    Write-Host "Performing backup actions for $webAppName..."
    foreach ($logFile in $logFilesToBackup) {
        $logFileName = $logFile.name
        $logFileUrl = $logFile.href
        Write-Host "Backing up log file: $logFileName"
        $blobName = "$webAppName/$($logFileUrl.Substring($logFileUrl.LastIndexOf("/") + 1))"
        Start-AzStorageBlobCopy -SrcUri $logFileUrl -DestContainer $containerName -DestBlob $blobName -Context $storageAccountContext
    }
    
    # Perform deletion actions...
    Write-Host "Deleting log files older than $numberOfDays days from $webAppName..."
    foreach ($logFile in $logFilesToBackup) {
        $logFileName = $logFile.name
        Write-Host "Deleting log file: $logFileName"
        Invoke-RestMethod -Uri $logFileUrl -Headers @{Authorization=("Basic $base64AuthInfo")} -Method DELETE
    }

Here is the warning message for copying to blob (nothing is copied)

WARNING: Ignore mismatch source storage context.. The source uri is https://wa01.scm.azurewebsites.net/api/vfs/LogFiles/kudu/, the end point is https://sa01.blob.core.windows.net/.

Here is the error for the delete action (need to force the delete activity)

Invoke-RestMethod : {"Message":"Cannot delete directory. It is either not empty or access is not allowed."}

The intention is to copy files and directories to storage account container and after copy delete from webapp

Can you try downloading logs into a temporary file path first and then upload it to a blob instead of directly pointing it out from Kudu. @SKumar — Jahnavi
– Jahnavi, Commented Apr 30, 2024 at 6:32

Jahnavi · Accepted Answer · 2024-04-30 11:03:14Z

To copy files and directories to storage account container and after copy delete from webapp using PowerShell:

To prevent URI mismatch warnings when copying log files from Kudu app log files, it's better to copy all the log files to a local path for easy accessibility. After downloading them, you can upload it to the blob storage container.

The detailed script is given below:

$webAppName  = "appsjl"
$resourceGroupName = "xxx"
$storageAccountName = "newaccs"
$containerName = "latest"
$numberOfDays = 90
$kuduApiUrl = "https://${webAppName}.scm.azurewebsites.net/api/vfs/LogFiles/"
Write-Host "Retrieving publishing profile for $webAppName..."
$publishingProfile = Get-AzWebAppPublishingProfile -ResourceGroupName $resourceGroupName -Name $webAppName
[xml]$publishingProfileXml = $publishingProfile
$publishingUserName = $publishingProfileXml.publishData.publishProfile[0].userName
$publishingPassword = $publishingProfileXml.publishData.publishProfile[0].userPWD
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes("${publishingUserName}:${publishingPassword}"))
Write-Host "Retrieving log files from $webAppName..."
$response = Invoke-RestMethod -Uri $uri -Headers $headers -Method GET
$dateTimeValues = $response | Where-Object { $_.mtime -ne $null } | ForEach-Object {
    $mtimeValue = $_.mtime.Split('.')[0]  # Remove milliseconds
    [DateTime]::ParseExact($mtimeValue, "yyyy-MM-dd'T'HH:mm:ss", [System.Globalization.CultureInfo]::InvariantCulture)
}

Script to download it into the local path:

foreach ($logFile in $response) {
$logFileName = $logFile.name
$logFileUrl = $logFile.href
$localpath = "<localfilepath to download the logfiles>"
Invoke-WebRequest -Uri $logFileUrl -Headers $headers -OutFile $localpath
$blobName = "$webAppName/$logFileName"
$key = "xxxxSe6Q=="
$context = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $key
Set-AzStorageBlobContent -File $localpath -Container $containerName -Blob $blobName -Context $context

Output:

Downloaded in local folder path:

enter image description here

Storage blob:

enter image description here

And for deleting them, the command you used is appropriate. Make sure all the parameters have been giving correctly and try executing the same command.

Reference: Set-AzStorageBlobContent

Thanks, in my scenario, downloading is not an option. Also, could you please help me with getting files recursively and delete them
To delete them recursively, use foreach loop or any other loop methods. @SKumar

Collectives™ on Stack Overflow

Start-AzStorageBlobCopy in powersell

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related