0

I currently have a bash script that runs some az commands to clean a storage account container and upload the contents of a directory to it:

az storage blob delete-batch --account-name $ACCOUNT_NAME --source $web
az storage blob upload-batch --account-name $ACCOUNT_NAME -s $SOURCE_PATH -d $web

I would like to reuse that functionality inside a Powershell Azure task that runs on Azure DevOps Services because I have a lot of other stuff going on that script besides the storage cleaning and upload.

What's the best way to migrate this? Been looking in the Powershell Azure module documentation but I can't find a proper equivalent to blob delete-batch and blob upload-batch.

Also though in calling the az command directly but for that I would have to login so I would need a way to pass the service principal details from the Powershell Azure task into the az login command before executing those lines.

Any ideas are welcome. thanks in advance

1

2 Answers 2

1

use Azure PowerShell to login Azure with service principal

You can use the following script

$appId = "your application id "
$password = "your application secret"
$secpasswd = ConvertTo-SecureString $password -AsPlainText -Force
$mycreds = New-Object System.Management.Automation.PSCredential ($appId, $secpasswd)

Add-AzureRmAccount -Credential $mycreds -Tenant "your tenat id" -ServicePrincipal

I can use Azure CLI command "az storage blob upload-batch" to upload a local directory to Azure storage. How to implement it with Azure PowerShell

Azure PowerShell does not provide the command like that. It just provides the command Set-AzureStorageBlobContent to allow customers to upload file to Azure storage. So you need to write a script with the command to implement how to upload a directory to Azure storage. For example

$StorageAccountKey=" "
$sourceFileRootDirectory=" "
$StorageAccountName=" "
$ContainerName=" "
$ctx = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
$container = Get-AzureStorageContainer -Name $ContainerName -Context $ctx

 if ($container) {
        $filesToUpload = Get-ChildItem $sourceFileRootDirectory -Recurse -File

        foreach ($x in $filesToUpload) {
            $blobName = ($x.fullname.Substring($sourceFileRootDirectory.Length + 1)).Replace("\", "/")

            Set-AzureStorageBlobContent -File $x.fullname -Container $container.Name -Blob $blobName -Context $ctx -Force:$Force 
        }
}

I can use Azure CLI command "az storage blob delete-batch" to clean up a container. How to implement it with Azure PowerShell Azure PowerShell does not provide the command that we can use to directly delete all blob in one container. So we need to write a script to implement it. Now, we have two choices

  1. Delete the container and create a new container with the same name
$StorageAccountKey=" "
$StorageAccountName=" "
$ContainerName=" "
$context = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
        Remove-AzureStorageContainer -Name $ContainerName -Context $context
        New-AzureStorageContainer -Name $ContainerName -Context $context
  1. Get all blobs in the container then delete them
$StorageAccountKey=" "
$StorageAccountName=" "
$ContainerName=" "
$Token = $null
$Total = 0
$MaxCount=5000
$context = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
do
 { 
     $Blobs = Get-AzureStorageBlob -Container $ContainerName -MaxCount $MaxCount  -ContinuationToken $Token -Context $context
     if($Blobs.Length -le 0) { Break;}
     $Token = $Blobs[$blobs.Count -1].ContinuationToken;

     foreach($blob in $blobs){
        Remove-AzStorageBlob -Blob $blob.Name -Container $ContainerName -Context $context

     }


 }
 While ($Token -ne $null)  
Sign up to request clarification or add additional context in comments.

1 Comment

I'll accept this answer since it fulfills all requirements. Unfortunately as I thought there's really no way to batch upload or delete the blobs with Azure Powershell. However I'm testing this task with AzCopy commands, for better performance of the uploads at least. Thanks for the help @Jim Xu
0

Why not use the Azure CLI task in Azure DevOps that uses a Service connection for the authentication part? See the documentation here.

3 Comments

Because I have other steps in the Powershell script that I don't want to rewrite to fully work with the Az CLI task, as I mentioned in the initial question
You could split the script you are working on to Part A (PS), Part B (CLI) and Part C (PS) ??
if there are no other better ways to solve this... yes. refactoring to multiple steps is the next approach

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.