Python code on Azure Function app to convert csv files to excel on Azure storage account

Question

I'm new to Python code and had to write it for a project at work. If any one help me to correct the below code i will be appreciated.

I have a Azure Storage account with several containers. I am using HttpTrigger Function to fire the Azure function app on Azure Data Factory. When the pipeline on ADF creates a CSV file , pipeline should be able to run the function app to get the filename as parametric and convert csv file to excel file on Azure Storage Account.

I run the code succesfully on my local but when i deploy it to Function app i get the error. I have been trying to run the code for weeks but without success.

--- __init__.py file

import os, uuid
from azure.storage.blob import  BlobServiceClient, BlobClient, ContainerClient
import pandas as pd
from io import StringIO,BytesIO
import azure.functions as func
import xlsxwriter
import logging
def main(req: func.HttpRequest) -> func.HttpResponse:
    logging.info('Python HTTP trigger function processed a request.')
    name= "1893_Item_20220206_err.csv" #'"' ,req.params.get('name'),'"'
    if name:    
        connect_str = os.getenv('AzureWebJobsStorage') 
        blob_service_client = BlobServiceClient.from_connection_string(connect_str)
        container_name = "test"
        blob_client = blob_service_client.get_blob_client(container=container_name, blob=name)
        excel_blob_client = blob_service_client.get_blob_client(container=container_name, blob="1893_Item_20220206_err.xlsx")
        blob = blob_client.download_blob()
        read_file = pd.read_csv (StringIO(blob.content_as_text()),sep="|")
        print(read_file)
        output = BytesIO()
        writer = pd.ExcelWriter(output, engine='xlsxwriter')
        read_file.to_excel (writer)
        writer.save()
        output.seek(0)
        workbook = output.read()
        excel_blob_client.upload_blob(workbook, overwrite=True)
    else:   
     return func.HttpResponse(
            "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",
             status_code=200
  )


-- function.json file
{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "authLevel": "anonymous",
      "type": "httpTrigger",
      "path":"test/{blobname}.csv",
      "direction": "in",
      "name": "req",
      "methods": [
        "get",
        "post"
      ]
    },
    {
      "type": "http",
      "direction": "out",
      "name": "$return"
    }
  ]
}

Hi and welcome to Stack Overflow. Please provide us with more relevant information to enable us to help you, like the actual error message you're getting. Only stating "it doesn't work" usually doesn't give enough info for us to be able to help you. Please refer to How to Ask. — rickvdbosch
– rickvdbosch, Commented Feb 21, 2022 at 7:49

rickvdbosch · Accepted Answer · 2022-02-21 07:59:54Z

Looking at your code my best guess would be that the value for the name parameter you've hardcoded contains the .csv extension while the name parameter in the path will not. You have that as a static string in your definition: "path":"test/{blobname}.csv".

Either append the csv to the name parameter, or remove it from the path specification.

Next to that, the parameter name in the path is blobname and not name.

Alternative solution

This scenario could be accomplished using an Azure Blob storage trigger for Azure Functions. That way the added blob will already be available as an InputStream.

The Blob storage trigger starts a function when a new or updated blob is detected. The blob contents are provided as input to the function.

Here's an example (taken from the article linked to above

Function.json:

{
    "scriptFile": "__init__.py",
    "disabled": false,
    "bindings": [
        {
            "name": "myblob",
            "type": "blobTrigger",
            "direction": "in",
            "path": "samples-workitems/{name}",
            "connection":"MyStorageAccountAppSetting"
        }
    ]
}

and

import logging
import azure.functions as func


def main(myblob: func.InputStream):
    logging.info('Python Blob trigger function processed %s', myblob.name)

Collectives™ on Stack Overflow

Python code on Azure Function app to convert csv files to excel on Azure storage account

1 Answer 1

Alternative solution

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Alternative solution

Comments

Your Answer

Sign up or log in

Post as a guest

Related