Dash(Python) - can't display dataframe in datatable after calculations

Question

I'm reading a text file(.csv mostly) from dash_core_components.Upload. I had no problems printing the file that i've taken. But, problems arise when i do some calculations and try printing that.
and the error is:

dash.exceptions.InvalidCallbackReturnValue: 
The callback for property `children`
of component `dataframe_output` returned a value
which is not JSON serializable.

In general, Dash properties can only be
dash components, strings, dictionaries, numbers, None,
or lists of those.

Here's what I've done and tried:

# importing required libraries
import dash
import dash_table
import pandas as pd
import dash_core_components as dash_core
import dash_html_components as dash_html
from dash.dependencies import Input, Output

# starting app layout
app.layout = dash_html.Div([
    # upload button to take csv files
    dash_core.Upload(id='upload_data',
               children=dash_html.Div(['Drag and Drop or ',
                                  dash_html.A('Select Files')
                                  ]),
               style={'width': '100%',
                      'height': '60px',
                      'lineHeight': '60px',
                      'borderWidth': '1px',
                      'borderStyle': 'dashed',
                      'borderRadius': '5px',
                      'textAlign': 'center',
                      'margin': '10px'
                      },
               multiple=False),
    # a 'Div' to return table output to
    dash_html.Div(id='dataframe_output'),
])

# callback to take and output the uploaded file
@app.callback(Output('dataframe_output', 'children'),
              [Input('upload_data', 'contents'),
               Input('upload_data', 'filename')])
def update_output(contents, filename):
    if contents is not None:
        # reading the file
        input_data = pd.read_csv(filename)
        # creating a dataframe that has info about "data types", "count of nulls", "count of unique values"
        info_dataframe = pd.concat([pd.DataFrame(input_data.dtypes,       columns=["data_types"]),
                                    pd.DataFrame(input_data.isna().sum(), columns=["count of blanks"]),
                                    pd.DataFrame(input_data.nunique(),    columns=["count of unique values"])
                                           ],
                                          axis=1, sort=True)
        # adding index as a row
        info_dataframe.reset_index(level=0, inplace=True)
        # returning it to 'Div'
        return dash_html.Div([
            dash_table.DataTable(
                id='table',
                columns=[{"name": i, "id": i} for i in info_dataframe .columns],
                # columns=[{"name": i, "id": i} for i in input_data.columns], # this works fine
                data=info_dataframe .to_dict("rows"),
                # data=input_data.to_dict("rows"), # this works fine
                style_cell={'width': '50px',
                            'height': '30px',
                            'textAlign': 'left'}
                        )
        ])

# running the app now
if __name__ == '__main__':
    app.run_server(debug=True, port=8050)

(I also want to save this to a text file after displaying on browser. how do i do that too).

First off, what I am Immediately wondering about is why you are sending a Div filled with a DataTable() to another Div (dataframe_output) in the first place. Doesn't it seem a little redundant? Try sending just the DataTable() to the placeholder Div. Secondly, as the previous is just my intuition, the way DataTable()'s are normally populated currently is via creating an empty one with the right column names/id's preset in, and then sending to that DataTable()'s data property a pandas dataframe expressed in dict() form - ex. df.to_dict(orient='rows'). — Coolio2654
– Coolio2654, Commented Nov 22, 2019 at 23:53
As an aside, the Dash error of ... returned a value which is not JSON serializable is really as simple as it looks - make sure all your callbacks only send normal outputs to wherever they're needed (and I suspect sending a Div is what confused Dash). Lastly, if you want to keep your dataframe in a temporary location for storing purposes, as FBruzzesi shows, he describes the old way users had to store memory in their apps, which still works, but you should look into a newer component that was created to help do that, community.plot.ly/t/announcing-the-storage-component/13758 . — Coolio2654
– Coolio2654, Commented Nov 23, 2019 at 0:00

FBruzzesi · Accepted Answer · 2019-11-22 10:35:51Z

This always worked for me - try using a hidden Div for storing json serialized dataframe

import dash
import dash_table
import pandas as pd
import dash_core_components as dash_core
import dash_html_components as dash_html
from dash.dependencies import Input, Output
import base64
import io

# starting app layout
app.layout = dash_html.Div([
    # upload button to take csv files
    dash_core.Upload(id='upload_data',
               children=dash_html.Div(['Drag and Drop or ',
                                  dash_html.A('Select Files')
                                  ]),
               style={'width': '100%',
                      'height': '60px',
                      'lineHeight': '60px',
                      'borderWidth': '1px',
                      'borderStyle': 'dashed',
                      'borderRadius': '5px',
                      'textAlign': 'center',
                      'margin': '10px'
                      },
               multiple=False),
    # Div to store json serialized dataframe
    dash_html.Div(id='json_df_store', style={'display':'none'}),
    # a 'Div' to return table output to
    dash_html.Div(id='dataframe_output'),
])

@app.callback(Output('json_df_store', 'children'),
              [Input('upload_data', 'contents'),
               Input('upload_data', 'filename')])
def load_df(content, filename):
    if content:
        # Modify the read_csv callback part
        content_type, content_string = contents.split(',')
        decoded = base64.b64decode(content_string)

        try:
            if 'csv' in filename:

                # Assume that the user uploaded a CSV file
                input_data  = pd.read_csv(io.StringIO(decoded.decode('utf-8')))

                info_dataframe = pd.DataFrame(data={
                                              "data_types": input_data.dtypes,
                                              "blanks_count": input_data.isna().sum(),
                                              "unique_count": input_data.nunique()
                                                   })

                # adding index as a row
                info_dataframe.reset_index(level=0, inplace=True)
                info_dataframe.rename(columns={'index':'col_name'}, inplace=True)

                info_dataframe['data_types'] = info_dataframe['data_types'].astype(str)

                return info_dataframe.to_json(date_format='iso', orient='split')

        except Exception as e:
            #print(e)
            return pd.DataFrame(data={'Error': e}, index=[0]).to_json(date_format='iso', orient='split')



# callback to take and output the uploaded file
@app.callback(Output('dataframe_output', 'children'),
              [Input('json_df_store', 'children')])
def update_output(json_df):

   info_dataframe = pd.read_json(json_df, orient='split')

   data = info_dataframe .to_dict("rows")
   cols = [{"name": i, "id": i} for i in info_dataframe .columns]

   child = dash_html.Div([
                dash_table.DataTable(
                                id='table',
                                data=data, 
                                columns=cols,
                                style_cell={'width': '50px',
                                            'height': '30px',
                                            'textAlign': 'left'}
                                      )
                           ])
    return child

# running the app now
if __name__ == '__main__':
    app.run_server(debug=True, port=8050)

that worked. but, I don't get why we would need a hidden Div for a thing as simple as showing a dataframe.
also, you commented link to an article(or thread) on your previous answer about why dash accepts files only from the local directory. Asking you now, because you have deleted that answer and comments went with it.
Hey @NaveenKumar, here the discussion link community.plot.ly/t/get-file-path-from-upload-component/9934/2 where others had a similar issue. I never had the necessity, thus I cannot help in such regard.

Collectives™ on Stack Overflow

Dash(Python) - can't display dataframe in datatable after calculations

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related