Use Numpy to Dynamically Create Arrays

Question

I am trying to use numpy to dynamically create a set of zeros based on the size of a separate numpy array.

This is a small portion of the code of a much larger project. I have posted everything relevant in this question. I have a function k means which takes in a dataset (posted below) and a k value (which is 3, for this example). I create a variable centroids which is supposed to look something like

[[4.9 3.1 1.5 0.1]
[7.2 3.  5.8 1.6]
[7.2 3.6 6.1 2.5]]

From there, I need to create a numpy array of "labels", one corresponding to every row in the dataset, of all zeroes with the same shape as the centroids array. Meaning, for a dataset with 5 rows, it would look like:
[[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
This is what I am trying to achieve, albiet on a dynamic scale (i.e. where the # of rows and columns in the dataset are unknown).

The following (hard coded, non numpy) satisfies that (assuming there are 150 lines in the dataset:

def k_means(dataset, k):
    centroids = [[5,3,2,4.5],[5,3,2,5],[2,2,2,2]]

    cluster_labels = []
    for i in range(0,150):
        cluster_labels.append([0,0,0,0])
    print (cluster_labels)

I am trying to do this dynamically with the following:

def k_means(dataset, k):
    centroids = dataset[numpy.random.choice(dataset.shape[0], k, replace=False), :]
    print(centroids)

    cluster_labels = []
    cluster_labels = numpy.asarray(cluster_labels)
    for index in range(len(dataset)):
        # temp_array = numpy.zeros_like(centroids)
        # print(temp_array)
        cluster_labels = cluster_labels.append(cluster_labels, numpy.zeros_like(centroids))

The current result is: AttributeError: 'numpy.ndarray' object has no attribute 'append'
Or, if I comment out the cluster_labels line and uncomment the temp, I get:

[[0. 0. 0. 0.]
[0. 0. 0. 0.]
[0. 0. 0. 0.]]

I will ultimately get 150 sets of that.

Sample of Iris Dataset:

5.1 3.5 1.4 0.2
4.9 3   1.4 0.2
4.7 3.2 1.3 0.2
4.6 3.1 1.5 0.2
5   3.6 1.4 0.2
5.4 3.9 1.7 0.4
4.6 3.4 1.4 0.3
5   3.4 1.5 0.2
4.4 2.9 1.4 0.2
4.9 3.1 1.5 0.1
5.4 3.7 1.5 0.2
4.8 3.4 1.6 0.2
4.8 3   1.4 0.1
4.3 3   1.1 0.1
5.8 4   1.2 0.2
5.7 4.4 1.5 0.4
5.4 3.9 1.3 0.4
5.1 3.5 1.4 0.3
5.7 3.8 1.7 0.3
5.1 3.8 1.5 0.3
5.4 3.4 1.7 0.2
5.1 3.7 1.5 0.4
4.6 3.6 1   0.2
5.1 3.3 1.7 0.5
4.8 3.4 1.9 0.2
5   3   1.6 0.2
5   3.4 1.6 0.4
5.2 3.5 1.5 0.2
5.2 3.4 1.4 0.2
4.7 3.2 1.6 0.2
4.8 3.1 1.6 0.2
5.4 3.4 1.5 0.4
5.2 4.1 1.5 0.1
5.5 4.2 1.4 0.2

Can anybody help me dynamically use numpy to achieve what I am aiming for?

Thanks.

A list has an append method, a numpy array (ndarray) does not. It's best to collect values in a list (the original cluster_labels) with list append, and create an array from that afterwards (outside the loop). — hpaulj
– hpaulj, Commented Feb 25, 2019 at 0:53
Hi @hpaulj, thank you for that, frankly I did not know the difference. Can you help to draft up a solution that implements your comment? — artemis
– artemis, Commented Feb 25, 2019 at 0:54
@JerryM. What is confusing me in your question is that your desired output for a dataset of 5 rows is [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]. Shouldn't it be (with arr = np.zeros_like(centroids) [arr, arr, arr, arr, arr], a 3D array? — iz_
– iz_, Commented Feb 25, 2019 at 1:16
As I mentioned, the zeros_like does not give me the result I want. I did post my attempt, though — artemis
– artemis, Commented Feb 25, 2019 at 1:21

Alperen · Accepted Answer · 2019-02-25 01:44:00Z

2

shape of a numpy array is the size of the array. In a 2D array shape represents (number of rows, number of columns). So, shape[0] is the number of rows and shape[1] is the number of columns. You can use numpy.zeros((dataset.shape[0], centroids.shape[1])) to create a numpy array with your desired dimensions. Here is an example code with modified version of your k-means function.

import numpy

def k_means(dataset, k):
    centroids = dataset[numpy.random.choice(dataset.shape[0], k, replace=False), :]
    print(centroids)

    cluster_labels = numpy.zeros((dataset.shape[0], centroids.shape[1]))
    print(cluster_labels)


dataset = numpy.array([[1,2,3,4,5,6,7,8,9,0], 
                    [3,4,5,6,4,3,2,2,6,7],
                    [4,4,5,6,7,7,8,9,9,0], 
                    [5,6,7,8,5,3,3,2,2,1],
                    [6,3,3,2,2,4,5,6,6,8]])

k_means(dataset, 2)

Output:

[[1 2 3 4 5 6 7 8 9 0]
 [5 6 7 8 5 3 3 2 2 1]]
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]

I used numpy.zeros((dataset.shape[0], centroids.shape[1])) to make it more similar to your code. Actually, numpy.zeros(dataset.shape) would do the same thing, because centroids.shape[1] and dataset.shape[1] is the same. The number of columns of centroids and the number columns dataset are the same, because you choose your centroids from the dataset. So, the last version should be like:

def k_means(dataset, k):
    centroids = dataset[numpy.random.choice(dataset.shape[0], k, replace=False), :]
    cluster_labels = numpy.zeros(dataset.shape)

edited Feb 25, 2019 at 1:44

answered Feb 25, 2019 at 1:23

Alperen

4,8525 gold badges32 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

artemis Over a year ago

Sorry, for this comment. I didn't write something correctly. This answered my question with a good explanation.

artemis Over a year ago

Hi, @Alperen. Is there a difference between [4.6 3.2 1.4 0.2] [5.8 2.8 5.1 2.4] [5.8 2.8 5.1 2.4] [5.8 2.8 5.1 2.4] [4.6 3.2 1.4 0.2] [6.8 3. 5.5 2.1] and [4.6 3.2 1.4 0.2], [5.8 2.8 5.1 2.4], [5.8 2.8 5.1 2.4], [5.8 2.8 5.1 2.4], [4.6 3.2 1.4 0.2], [6.8 3. 5.5 2.1]

Alperen Over a year ago

Because of the syntax of Python, you need to use commas while defining the array. Actually, you are defining array of arrays to define 2D array. So, you need commas between items.

artemis Over a year ago

the reason I ask is your solution does not yield commas, however the one that I posted as a "hack fix" did. But yours seems to run with my program?

Alperen Over a year ago

If I understand correctly, you are talking about printing a numpy array. It is all about the numpy package. I guess, they override str() function in numpy package for more understandable print outputs. For example, lets define a list a=[[1,2],[2,3]]. If you print it, it will yield [[1, 2], [3, 4]]. If you define it as numpy array a = numpy.array(a) and print it, it will yield [[1 2] [3 4]] in two rows. Try it yourself.

|

Collectives™ on Stack Overflow

Use Numpy to Dynamically Create Arrays

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related