How to convert this code based on lists to code based on numpy arrays?

Question

I am working with the keras and there is always a from with a lists, so I guess that always everything has to be converted to the numpy array which are very illogical for me. I guess that it is associated with performance? I don't see any other reason? However my problem looks like shown below. I have to convert this part of code:

output_sentence = []
final_output_sentence = []

for key in row['o'].lower():

    temp_list = []

    if key in dictionary.keys():

        temp_list.append(dictionary[key])
        output_sentence.append(temp_list)

    else:

        dictionary[key] = len(dictionary)
        temp_list.append(dictionary[key])
        output_sentence.append(temp_list)

final_output_sentence.append(output_sentence)

to the code based on numpy arrays. I try in this way:

output_sentence = np.array([], dtype=int)
final_output_sentence = np.array([], dtype=int)

for key in row['o'].lower():

    temp_list = np.array([], dtype=int)

    if key in dictionary.keys():

        temp_list = np.append(temp_list, dictionary[key])

        output_sentence = np.append(output_sentence, temp_list)

    else:

        dictionary[key] = len(dictionary)
        temp_list = np.append(temp_list, dictionary[key])
        output_sentence = np.append(output_sentence, temp_list)

final_output_sentence = np.append(final_output_sentence, output_sentence)

however instead of this [[[1], [2], [3], [2], [4]]] I get this [1 2 3 2 4]. Any ideas how to solve this?

UPDATE

What do you think about solution shown below? Any tips for performance optimization?

output_sentence = []

for key in row['o'].lower():

    temp_list = []

    if key in dictionary.keys():

        temp_list.append(dictionary[key])
        output_sentence.append(temp_list)

    else:

        dictionary[key] = len(dictionary)
        temp_list.append(dictionary[key])
        output_sentence.append(temp_list)

final_output_sentence = np.array(output_sentence)

final_output_sentence = final_output_sentence.reshape(1, final_output_sentence.shape[0], 1)

Generally it is better to use lists and list append, and create an array with one call at the end. np.append (and other versions of np.concatenate) are slower, and harder to apply correctly. — hpaulj
– hpaulj, Commented Mar 23, 2019 at 21:17
With a sample of row and dictionary, and the results from the list version, we might be able to suggest improvements. Sometimes there are ways of replacing list operations with whole-array ones. But replicating the list appends is not one of those. Why are you appending to 2 lists, when you only have one iteration loop (on key)? — hpaulj
– hpaulj, Commented Mar 23, 2019 at 21:22
@hpaulj Do you mean this two lines temp_list.append(dictionary[key]) output_sentence.append(temp_list)? — thedbogh
– thedbogh, Commented Mar 23, 2019 at 21:27

mujjiga · Accepted Answer · 2019-03-24 00:31:15Z

1

output_sentence = []
for key in row['o'].lower():
    if key not in dictionary.keys():
        dictionary[key] = len(dictionary)
    output_sentence.append(dictionary[key])

final_output_sentence = np.array(output_sentence).reshape(1,-1,1)

If the key does not exist in dictionary then add it with the next size
Append the value corresponding corresponding to the key into output_sentence
Finally, output_sentence is a list but since you want a 3D array, convert it into numpy array and reshape it.
x.reshape(1,-1,1) => reshape x such that size of 0th axis is 1, size of 2nd axis is 1 and the size of 1st axis will be same a elements in x.

edited Mar 24, 2019 at 0:31

answered Mar 23, 2019 at 23:39

mujjiga

17.1k2 gold badges37 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

mujjiga Over a year ago

@NinoFiliu Done will make sure to do so from now on.

Collectives™ on Stack Overflow

How to convert this code based on lists to code based on numpy arrays?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related