Create Google Cloud Function using API in Python

Question

I'm working on a project with Python(3.6) & Django(1.10) in which I need to create a function at Google cloud using API request.

How can upload code in the form of a zip archive while creating that function?

Here's what I have tried:

From views.py :

    def post(self, request, *args, **kwargs):
    if request.method == 'POST':
        post_data = request.POST.copy()
        post_data.update({'user': request.user.pk})
        form = forms.SlsForm(post_data, request.FILES)
        print('get post request')
        if form.is_valid():
            func_obj = form
            func_obj.user = request.user
            func_obj.project = form.cleaned_data['project']
            func_obj.fname = form.cleaned_data['fname']
            func_obj.fmemory = form.cleaned_data['fmemory']
            func_obj.entryPoint = form.cleaned_data['entryPoint']
            func_obj.sourceFile = form.cleaned_data['sourceFile']
            func_obj.sc_github = form.cleaned_data['sc_github']
            func_obj.sc_inline_index = form.cleaned_data['sc_inline_index']
            func_obj.sc_inline_package = form.cleaned_data['sc_inline_package']
            func_obj.bucket = form.cleaned_data['bucket']
            func_obj.save()
            service = discovery.build('cloudfunctions', 'v1', http=views.getauth(), cache_discovery=False)
            requ = service.projects().locations().functions().generateUploadUrl(parent='projects/' + func_obj.project + '/locations/us-central1', body={})
            resp = requ.execute()
            print(resp)
            try:
                auth = views.getauth()
                # Prepare Request Body
                req_body = {
                    "CloudFunction": {
                        "name": func_obj.fname,
                        "entryPoint": func_obj.entryPoint,
                        "timeout": '60s',
                        "availableMemoryMb": func_obj.fmemory,
                        "sourceArchiveUrl": func_obj.sc_github,
                    },
                    "sourceUploadUrl": func_obj.bucket,
                }
                service = discovery.build('cloudfunctions', 'v1beta2', http=auth, cachce_dicovery=False)
                func_req = service.projects().locations().functions().create(location='projects/' + func_obj.project
                                                                                      + '/locations/-',
                                                                             body=req_body)
                func_res = func_req.execute()
                print(func_res)
                return HttpResponse('Submitted',)
            except:
                return HttpResponse(status=500)

        return HttpResponse('Sent!')

Updated Code below:

            if form.is_valid():
            func_obj = form
            func_obj.user = request.user
            func_obj.project = form.cleaned_data['project']
            func_obj.fname = form.cleaned_data['fname']
            func_obj.fmemory = form.cleaned_data['fmemory']
            func_obj.entryPoint = form.cleaned_data['entryPoint']
            func_obj.sourceFile = form.cleaned_data['sourceFile']
            func_obj.sc_github = form.cleaned_data['sc_github']
            func_obj.sc_inline_index = form.cleaned_data['sc_inline_index']
            func_obj.sc_inline_package = form.cleaned_data['sc_inline_package']
            func_obj.bucket = form.cleaned_data['bucket']
            func_obj.save()

            #######################################################################
            # FIRST APPROACH FOR FUNCTION CREATION USING STORAGE BUCKET
            #######################################################################

            file_name = os.path.join(IGui.settings.BASE_DIR, 'media/archives/', func_obj.sourceFile.name)
            print(file_name)

            service = discovery.build('cloudfunctions', 'v1')
            func_api = service.projects().locations().functions()
            url_svc_req = func_api.generateUploadUrl(parent='projects/'
                                                            + func_obj.project
                                                            + '/locations/us-central1',
                                                     body={})
            url_svc_res = url_svc_req.execute()
            print(url_svc_res)

            upload_url = url_svc_res['uploadUrl']
            print(upload_url)
            headers = {
                'content-type': 'application/zip',
                'x-goog-content-length-range': '0,104857600'
            }
            print(requests.put(upload_url, headers=headers, data=func_obj.sourceFile.name))
            auth = views.getauth()
            # Prepare Request Body
            name = "projects/{}/locations/us-central1/functions/{}".format(func_obj.project, func_obj.fname,)
            print(name)
            req_body = {
              "name": name,
              "entryPoint": func_obj.entryPoint,
              "timeout": "3.5s",
              "availableMemoryMb": func_obj.fmemory,
              "sourceUploadUrl": upload_url,
              "httpsTrigger": {},
            }
            service = discovery.build('cloudfunctions', 'v1')
            func_api = service.projects().locations().functions()

            response = func_api.create(location='projects/' + func_obj.project + '/locations/us-central1',
                                                body=req_body).execute()

            pprint.pprint(response)

Now the function has been created successfully, but it fails because the source code doesn't upload to storage bucket, that's maybe something wrong at:

upload_url = url_svc_res['uploadUrl']
            print(upload_url)
            headers = {
                'content-type': 'application/zip',
                'x-goog-content-length-range': '0,104857600'
            }
            print(requests.put(upload_url, headers=headers, data=func_obj.sourceFile.name))

ostergaard · Accepted Answer · 2020-02-15 19:12:50Z

6

In the request body you have a dictionary "CloudFunction" inside the request. The content of "CloudFunction" should be directly in request.

request_body = {
    "name": parent + '/functions/' + name,
    "entryPoint": entry_point,
    "sourceUploadUrl": upload_url,
    "httpsTrigger": {}
}

I recomend using "Try this API" to discover the structure of projects.locations.functions.create .

"sourceArchiveUrl" and "sourceUploadUrl" can't appear together. This is explained in Resorce Cloud Function:

// Union field source_code can be only one of the following:
"sourceArchiveUrl": string,
"sourceRepository": { object(SourceRepository) },
"sourceUploadUrl": string,
// End of list of possible types for union field source_code.

In the rest of the answer I assume that you want to use "sourceUploadUrl". It requires you to pass it a URL returned to you by .generateUploadUrl(...).execute(). See documentation:

sourceUploadUrl -> string

The Google Cloud Storage signed URL used for source uploading, generated by [google.cloud.functions.v1.GenerateUploadUrl][]

But before passing it you need to upload a zip file to this URL:

curl -X PUT "${URL}" -H 'content-type:application/zip' -H 'x-goog-content-length-range: 0,104857600'  -T test.zip

or in python:

    headers = {
        'content-type':'application/zip',
        'x-goog-content-length-range':'0,104857600'
    }
    print(requests.put(upload_url, headers=headers, data=data))

This is the trickiest part:

the case matters and it should be lowercase. Because the signature is calculated from a hash (here)
you need 'content-type':'application/zip'. I deduced this one logically, because documentation doesn't mention it. (here)
x-goog-content-length-range: min,max is obligatory for all PUT requests for cloud storage and is assumed implicitly in this case. More on it here
104857600, the max in previous entry, is a magical number which I didn't found mentioned anywhere.

where data is a FileLikeObject.

I also assume that you want to use the httpsTrigger. For a cloud function you can only choose one trigger field. Here it's said that trigger is a Union field. For httpsTrigger however that you can just leave it to be an empty dictionary, as its content do not affect the outcome. As of now.

request_body = {
    "name": parent + '/functions/' + name,
    "entryPoint": entry_point,
    "sourceUploadUrl": upload_url,
    "httpsTrigger": {}
}

You can safely use 'v1' instead of 'v1beta2' for .create().

Here is a full working example. It would be to complicated if I presented it to you as part of your code, but you can easily integrate it.

import pprint
import zipfile
import requests
from tempfile import TemporaryFile
from googleapiclient import discovery

project_id = 'your_project_id'
region = 'us-central1'
parent = 'projects/{}/locations/{}'.format(project_id, region)
print(parent)
name = 'ExampleFunctionFibonacci'
entry_point = "fibonacci"

service = discovery.build('cloudfunctions', 'v1')
CloudFunctionsAPI = service.projects().locations().functions()
upload_url = CloudFunctionsAPI.generateUploadUrl(parent=parent, body={}).execute()['uploadUrl']
print(upload_url)


payload = """/**
 * Responds to any HTTP request that can provide a "message" field in the body.
 *
 * @param {Object} req Cloud Function request context.
 * @param {Object} res Cloud Function response context.
 */
exports.""" + entry_point + """= function """ + entry_point + """ (req, res) {
  if (req.body.message === undefined) {
    // This is an error case, as "message" is required
    res.status(400).send('No message defined!');
  } else {
    // Everything is ok
    console.log(req.body.message);
    res.status(200).end();
  }
};"""


with TemporaryFile() as data:
    with zipfile.ZipFile(data, 'w', zipfile.ZIP_DEFLATED) as archive:
        archive.writestr('function.js', payload)

    data.seek(0)
    headers = {
        'content-type':'application/zip',
        'x-goog-content-length-range':'0,104857600'
    }
    print(requests.put(upload_url, headers=headers, data=data))

# Prepare Request Body
# https://cloud.google.com/functions/docs/reference/rest/v1/projects.locations.functions#resource-cloudfunction

request_body = {
    "name": parent + '/functions/' + name,
    "entryPoint": entry_point,
    "sourceUploadUrl": upload_url,
    "httpsTrigger": {},
    "runtime": 'nodejs8'
}

print('https://{}-{}.cloudfunctions.net/{}'.format(region,project_id,name))
response = CloudFunctionsAPI.create(location=parent, body=request_body).execute()

pprint.pprint(response)

Open and upload a zip file like following:

file_name = os.path.join(IGui.settings.BASE_DIR, 'media/archives/', func_obj.sourceFile.name)
headers = {
    'content-type': 'application/zip',
    'x-goog-content-length-range': '0,104857600'
}

with open(file_name, 'rb') as data:
    print(requests.put(upload_url, headers=headers, data=data))

edited Feb 15, 2020 at 19:12

ostergaard

3,5372 gold badges33 silver badges42 bronze badges

answered Nov 28, 2017 at 11:49

A.Queue

1,5928 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Abdul Rehman Over a year ago

Hi, @A.Queue ! first thanks for your answer. I have updated my code, see in question please! But now it returns as :

File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake     self._sslobj.do_handshake() ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:749) [28/Nov/2017 13:33:10] "POST /user/deploy/serverless/deployment/ HTTP/1.1" 500 17239

and even file doesn't uploaded on bucket but returns 200 status code.

Abdul Rehman Over a year ago

Now function creation is working but the source code doesn't upload to storage bucket!

A.Queue Over a year ago

Is this the entire traceback? Seems like it's a domino effect. So far you have file_name = os.path.join(IGui.settings.BASE_DIR, 'media/archives/', func_obj.sourceFile.name) but you never use it. Also you do requests.put(upload_url, headers=headers, data=func_obj.sourceFile.name) but I am not sure if func_obj.sourceFile.name returns FileLikeObject or a string with the name. I also recommend isolating this code completely into a different function so that you can isolate it from rest of the django. Chances are this error is caused by another error.

Abdul Rehman Over a year ago

this is the path to the uploaded file: file_name = os.path.join(IGui.settings.BASE_DIR, 'media/archives/', func_obj.sourceFile.name)

A.Queue Over a year ago

To return a FileLikeObject you need to do with open(file_name, 'r') as tmp construction like in my code. Then pass it to data like data=tmp.

|

Collectives™ on Stack Overflow

Create Google Cloud Function using API in Python

1 Answer 1

9 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related