Deploying a Plotly/Dash app to AWS using Serverless Framework

Question

I am trying to deploy a Plotly Dash app as an AWS Lambda using Serverless framework. The app works as expected locally and I can start it using serverless wsgi serve command. serverless deploy reports success. However when invoked, lambda fails with the following error:

Traceback (most recent call last):
File "/var/task/wsgi_handler.py", line 44, in import_app
wsgi_module = importlib.import_module(wsgi_fqn_parts[-1])
File "/var/lang/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/var/task/app.py", line 4, in <module>
import dash
File "/tmp/sls-py-req/dash/__init__.py", line 5, in <module>
from .dash import Dash, no_update  # noqa: F401,E402
File "/tmp/sls-py-req/dash/dash.py", line 21, in <module>
from flask_compress import Compress
File "/tmp/sls-py-req/flask_compress.py", line 14, in <module>
import brotli
File "/tmp/sls-py-req/brotli.py", line 8, in <module>
import _brotli
ModuleNotFoundError: No module named '_brotli'
[ERROR] Exception: Unable to import app.server
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/var/lang/lib/python3.8/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/task/wsgi_handler.py", line 119, in <module>
    wsgi_app = import_app(config)
  File "/var/task/wsgi_handler.py", line 49, in import_app
    raise Exception("Unable to import 
{}
".format(config["app"]))

app.py

import dash
import dash_html_components as html
from flask import Flask

server = Flask(__name__)

@server.route("/")
def index():
    return "Hello Flask app"

app = dash.Dash(
    __name__,
    server=server,
    routes_pathname_prefix='/dash/'
)

app.layout = html.Div(html.H1("Hello Dash!"))

if __name__ == "__main__":
    server.run(debug=True)

serverless.yml

---
service: dash-serverless

variablesResolutionMode: 20210219
useDotenv: true

provider:
  name: aws
  runtime: python3.8
  stage: test
  region: eu-central-1
  apiGateway:
    shouldStartNameWithService: true
  lambdaHashingVersion: 20201221

functions:
  app:
    handler: wsgi_handler.handler
    events:
      - http: ANY /
      - http: "ANY {proxy+}"

custom:
  wsgi:
    app: app.server
    pythonBin: python3
    packRequirements: false
  pythonRequirements:
    dockerPip: non-linux

plugins:
  - serverless-wsgi
  - serverless-python-requirements

package:
  exclude:
    - node_modules/**
    # and other

requirments.txt

-i https://pypi.org/simple
brotli==1.0.9
click==7.1.2; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'
dash-core-components==1.15.0
dash-html-components==1.1.2
dash-renderer==1.9.0
dash-table==4.11.2
dash==1.19.0
flask-compress==1.9.0
flask==1.1.2
future==0.18.2; python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'
itsdangerous==1.1.0; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
jinja2==2.11.3; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'
markupsafe==1.1.1; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
plotly==4.14.3
retrying==1.3.3
six==1.15.0; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
werkzeug==1.0.1; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'

Sergii Gryshkevych · Accepted Answer · 2021-04-05 14:20:12Z

The reason for ModuleNotFoundError: No module named '_brotli' is improper dependencies packaging. It is fixed by packaging the app via the use of Docker and the docker-lambda image. slim: true and strip: false minimise the package size while preserving binaries wich is required in some cases (in this example it does).

pythonRequirements:
    dockerizePip: true
    slim: true
    strip: false

Having solved the _brotli not found issue, I encountered next error message pkg_resources.DistributionNotFound: The 'flask-compress' distribution was not found and is required by the application. I was able to solve it with a workaround described in Pyinstaller executable cannot find 'flask-compress' distribution that is included.

Finally, the application is served under /<api-stage-name>/ (/test/ in this example). This requires a modification the Dash app configuration, namely providing requests_pathname_prefix="/test/dash/" in the Dash constructor.

Full working example:

app.py

from collections import namedtuple
import pkg_resources

# backup true function
_true_get_distribution = pkg_resources.get_distribution
# create small placeholder for the dash call
# _flask_compress_version = parse_version(get_distribution("flask-compress").version)
_Dist = namedtuple('_Dist', ['version'])


def _get_distribution(dist):
    if dist == 'flask-compress':
        return _Dist('1.9.0'). # your flask-compress version
    else:
        return _true_get_distribution(dist)


# monkey patch the function so it can work once frozen and pkg_resources is of
# no help
pkg_resources.get_distribution = _get_distribution

import dash
import dash_html_components as html
from flask import Flask


server = Flask(__name__)


@server.route("/")
def index():
    return "Hello Flask app"


app = dash.Dash(
    __name__,
    server=server,
    routes_pathname_prefix="/dash/",
    requests_pathname_prefix="/test/dash/"
)

app.layout = html.Div(html.H1("Hello Dash!"))

if __name__ == "__main__":
    server.run(debug=True)

serverless.yml

---
service: dash-serverless

variablesResolutionMode: 20210219
useDotenv: true

provider:
  name: aws
  runtime: python3.8
  stage: test
  region: eu-central-1
  apiGateway:
    shouldStartNameWithService: true
  lambdaHashingVersion: 20201221

functions:
  app:
    handler: wsgi_handler.handler
    events:
      - http: ANY /
      - http: "ANY {proxy+}"

custom:
  wsgi:
    app: app.server
    pythonBin: python3
    packRequirements: false
  pythonRequirements:
    dockerizePip: true
    slim: true
    strip: false

plugins:
  - serverless-wsgi
  - serverless-python-requirements

package:
  exclude:
    - node_modules/**
    # and other

Collectives™ on Stack Overflow

Deploying a Plotly/Dash app to AWS using Serverless Framework

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related