Prevent adding duplicate data from excel forms imported into db

Question

I have a model on which data imported from an excel files are saved. I want to prevent having duplicate entries,it will check the already existing data to see if they match with the ones .

My model

from django.db import models


class UserData(models.Model):
    GENDER_CHOICES = (
        ('Male', 'Male'),
        ('Female', 'Female'),
    )

    first_name = models.CharField(max_length=50)
    last_name = models.CharField(max_length=50)
    age = models.IntegerField()
    gender = models.CharField(default='Male', choices=GENDER_CHOICES, max_length=6)
    address = models.CharField(max_length=200)

    class Meta:
        verbose_name_plural = 'User Data'

    def __str__(self):
        return self.fullname()

views.py

class UploadFileForm(forms.Form):
    file = forms.FileField()


def import_data(request):
    if request.method == "POST":
        form = UploadFileForm(request.POST,
                          request.FILES)
        if form.is_valid():
            request.FILES['file'].save_to_database(
                name_columns_by_row=2,
                model=UserData,
                mapdict=['first_name', 'last_name', 'age', 'gender', 'address'])


        return HttpResponse("OK")
    else:
        return HttpResponseBadRequest()
else:
    form = UploadFileForm()
return render_to_response('excel/upload_form.html',
                          {'form': form},
                          context_instance=RequestContext(request))

I tried using unique_together and also tried overriding cleaned_data in the form but still can't prevent duplicates being added to the db.

What will be the best way to achieve this? Thanks

At the moment no. But when I tried overriding the cleaned_data I got KeyError saying first_name doesn't exist when it clearly does. — Bernard 'Beta Berlin' Parah
– Bernard 'Beta Berlin' Parah, Commented Aug 25, 2016 at 10:51

Community · Accepted Answer · 2017-05-23 12:25:21Z

2

1.Create one model form

class UserDataForm(forms.ModelForm):
    // code here
    class Meta:
        model = UserData

2. make the excel sheet data to dictionary format using.

from pandas import *
xls = ExcelFile('path_to_file.xls')
df = xls.parse(xls.sheet_names[0])

ref: Python Creating Dictionary from excel data.

3.Instantiate the model form with the dictionary data and check for validation. if the data is valid check for the db entry for the same or use django model_name.get_or_create()

form = UploadFileForm(request.POST,
                      request.FILES)
    if form.is_valid():
        # code here to convert to dictionary 
        data = data_in_dict_form
        user_data_form = UserDataForm(data=data)
        if user_data_form.is_valid():
            obj, created = UserData.objects.get_or_create(**user_data_form.cleaned_data)
            # this will create new entry if doesnt exist and just get from db if already exist.
            # You can ause any method to create an entry to the data base. you can also just .filter().exists() to check if there is an existing entry. if It returns True you dont need to save it. 
            # Your Response
         else:
            # your Response

Note: If there are multiple rows , each rows are for separate entries in db. you need to loop each rows and validate and save()/Do_not_save. :)

I hope this would help you to get the problem resolved.

edited May 23, 2017 at 12:25

CommunityBot

11 silver badge

answered Aug 31, 2016 at 5:30

Vipul Vishnu av

5262 gold badges6 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Stop War Over a year ago

May be I am missing something obvious but still can't figure out data_in_dict_form in data = data_in_dict_form. Can this be clarified?

Collectives™ on Stack Overflow

Prevent adding duplicate data from excel forms imported into db

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related