Python Object List Update

Question

I have an object with a structure similar to:

models = [{
  'name': 'Model 1',
  'errors': {
    'mse': None,
    'rmse': None
  }
},
{
  'name': 'Model 2',
  'errors': {
    'mse': None,
    'rmse': None
  }
}]

Now I want to loop through the object and change something, so I tried using something like this:

for index, model in enumerate(models):
  # Carry out model performance and update the respective errors
  model['errors']['mse'] = metrics.mean_squared_error(...)

What happens is all the objects inside the models list gets updated with the last element's values. I tried using this as well (and it also didn't work):

for index in range(len(models)):
  # Carry out model performance and update the respective errors
  models[index]['errors']['mse'] = metrics.mean_squared_error(...)

If I even write models[0]['errors']['mse'] = 10, all the other indexes also gets updated. Can anyone please help me with a way around this issue?

(It'll really be great if you can suggest a solution where I can update while looping through the list)

Actual Code

import importlib

class Models:
    
    # Model structure
    model = {
        'base': None,
        'name': None,
        'model': None,
        'config': {},
        'errors': {
            'mae': None,
            'mse': None,
            'rmse': None,
        },
        'scores': {
            'r2': None,
            'accuracy_score': None
        }
    }
    
    # Prints Looping Progress
    def progress(self, index):
        return '(' + str(index+1) + '/' + str(self.models.__len__()) + ')'
    
    # Import Models
    def import_models(self):
        for model in self.models:
            model['model'] = getattr(importlib.import_module(model['base']), model['name'])(**model['config'])
    
    # Restructure Models Object
    def restructure_models(self, models):
        restructured_models = []
        for model in models:
            restructured_model = {**self.model, **model}
            restructured_models.append(restructured_model)
        return restructured_models
    
    # Constructor
    def __init__(self, models):
        self.models = self.restructure_models(models)
        self.import_models()
    
    # Train Models
    def fit(self, X_train, y_train):
        for index, model in enumerate(self.models):
            print('Training ' + self.progress(index) + ': ', model['name'])
            model['model'].fit(X_train, y_train)
        print('\n')
        
    # Predict Values
    def predict(self, X_test):
        all_predictions = []
        for index, model in enumerate(self.models):
            print('Running ' + self.progress(index) + ': ', model['name'])
            # Predict
            predictions = model['model'].predict(X_test)
            all_predictions.append({
                'model': model['name'],
                'predictions': predictions
            })
        print('\n')
        return all_predictions
            
    # Evaluate Trained Models
    def test(self, X_test, y_test):
        from sklearn import metrics
        all_predictions = []
        for index, model in enumerate(self.models):
            print('Evaluating ' + self.progress(index) + ': ', model['name'])
            # Predict
            predictions = model['model'].predict(X_test)
            # Errors
            model['errors']['mae'] = metrics.mean_absolute_error(y_test, predictions)
            model['errors']['mse'] = metrics.mean_squared_error(y_test, predictions)
            model['errors']['rmse'] = np.sqrt(model['errors']['mse'])
            # Scores
            model['scores']['r2'] = metrics.r2_score(y_test, predictions)
            model['scores']['accuracy_score'] = metrics.r2_score(y_true = y_test, y_pred = predictions)
            all_predictions.append({
                'model': model['name'],
                'predictions': predictions
            })
        print('\n')
        return all_predictions
    
    # Evaluated Performance Metrics
    def results(self):
        for model in self.models:
            print('Model: ', model['name'])
            print('MAE: ', model['errors']['mae'])
            print('MSE: ', model['errors']['mse'])
            print('RMSE: ', model['errors']['rmse'])
            print('R2: ', model['scores']['r2'])
            print('Accuracy Score: ', model['scores']['accuracy_score'])
            print('\n')

Calling and using the class:

selected_models = [
    {
        'base': 'sklearn.linear_model',
        'name': 'LinearRegression'
    },
    {
        'base': 'sklearn.ensemble',
        'name': 'RandomForestRegressor',
                'config': {
                        'n_estimators': 50
                }
    }
]
models = Models(models = selected_models)
predictions = models.test(X_test = X_test_scaled, y_test = y_test)

The problem is with the test function updating the self.models incorrectly (all list items are changed according to the last element)

Please post code that actually demonstrates the problem -- or try to reproduce the problem with the sample code you've provided where each list element is an independently declared dict (you'll find that it behaves differently). — Samwise
– Samwise, Commented May 31, 2021 at 14:13

quamrana · Accepted Answer · 2021-05-31 14:35:47Z

2

The problem is here:

restructured_model = {**self.model, **model}

You are making a new dict, but it references the singleton Models.model.

I think you meant this:

import copy

copy_model = copy.deepcopy(self.model)
restructured_model = {**copy_model, **model}

answered May 31, 2021 at 14:35

quamrana

39.5k13 gold badges57 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Dragonborn · Accepted Answer · 2021-05-31 14:21:51Z

1

Can you try this for debug?

for index, model in enumerate(models):
  # Carry out model performance and update the respective errors
  set_value = metrics.mean_squared_error(...)
  print('setting value to ' + str(set_value))
  model['errors']['mse'] = set_value

answered May 31, 2021 at 14:21

Dragonborn

1,8151 gold badge18 silver badges37 bronze badges

1 Comment

bacon-delight Over a year ago

I added it, and the results are again the same. While printing inside the loop, it gives the correct values, however after running the loop when I print the entire object, all the items are updated with the last element's value.

Collectives™ on Stack Overflow

Python Object List Update

Actual Code

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Actual Code

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related