Dealing with Irregular Embedded ArrayFields in Django

Question

I am currently using Django with PostgreSQL. I use django-extensions to run custom scripts, inserting data to database. I have a model as below:

class Route(models.Model):
    """
    Bus routes of İzmir.
    """

    code = models.PositiveSmallIntegerField(
        unique=True,
        primary_key=True,
        verbose_name="Code"
    )

    stops = models.ManyToManyField(
        Stop,
        null=True,
        blank=True,
        related_name="routes",
        verbose_name="Stops"
    )

    terminals = ArrayField(
        models.CharField(
            null=False,
            blank=False,
            max_length=32,
        ),
        size=2,
        default=[],
        verbose_name="Terminals"
    )

    departure_times = ArrayField(
        ArrayField(
            models.TimeField(
                null=False,
                blank=False
            ),
            null=True, # This was added later to maybe solve the problem.
            default=[]
        ),
        default=[],
        size=6,
        verbose_name="Departure Times"
    )

    class Meta:
        verbose_name = "Route"
        verbose_name_plural = "Routes"
        ordering = ["code"]

As you can see, I have embedded another ArrayField to departure_times. My data has irregular shapes as mentioned in Django documentation as a note. So, naturally, it raises the error as below:

django.db.utils.DataError: multidimensional arrays must have array expressions with matching dimensions

As mentioned in documentation as "the underlying field should be made nullable", I added null = True to my embedded ArrayField. However, I did not understand the following statement in the note in documentation:

the values padded with None

The questions on my head are

1) Will I deal with those data, modifying the length of irregular lists as below?

[a, b, c, None, None]
[a, b, None, None, None]
[a, b, c, d, e]

2) Is there a built-in way to overcome DataError?

If you think seeing the script would help you to solve the problem, here it is.

Environment

python 3.5.1
django 1.9.7
psycopg2 2.6.2
postgresql 9.5

Eray Erdin · Accepted Answer · 2016-07-14 12:38:22Z

So, there seems to be no participant to this question. Thus, I wanted to share my own solution. In short, I modified the script in a way of question (1) above, which means, I extended each list with filler None values to equalize the length of every list.

Assuming I have data as below:

[
    [1, 2, 3],
    [4, 5],
    [6, 7, 8, 9, 10]
]

Desired output is:

[
    [1, 2, 3, None, None],
    [4, 5, None, None, None],
    [6, 7, 8, 9, 10]
]

So, I've just created a sample function in case anyone needs in future, as below:

def equalizer(data):
    largest_length = 0 # To define the largest length

    for l in data:
        if len(l) > largest_length:
            largest_length = len(l) # Will define the largest length in data.

    for i, l in enumerate(data):
        if len(l) < largest_length:
            remainder = largest_length - len(l) # Difference of length of particular list and largest length
            data[i].extend([None for i in range(remainder)]) # Add None through the largest length limit

    return data

Then there is not any DataError to show. This answer will be accepted as valid in two days if there are not any better solution.

Max Smith · Accepted Answer · 2017-01-19 23:04:54Z

Thank you for sharing your solution it really helped me out. I believe however that I found a more elegant solution. He's my take on it:

    def save(self, *args, **kwargs):
         max_lenght = max(len(element) for element in self.NestedArrays)
         [[element.append(None) for _ in range(max_lenght - len(element))] for element in self.NestedArrays if len(element) != max_lenght]
         super(ParentClass, self).save(*args, **kwargs)  # Call the "real" save() method.

Basically I'm overriding the default save method. Then modifying the NestedArrays attribute and finally saving it. This method would be added to the Model class that had this issue. In this case I called it ParentClass.

More info on overriding model methods here.

Hope this helps someone!

Collectives™ on Stack Overflow

Dealing with Irregular Embedded ArrayFields in Django

The questions on my head are

Environment

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

The questions on my head are

Environment

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related