Keras Hyperparameter Tuning with Functional API

Question

I am trying to user Keras Tuner to optimize some hyperparameters, and this is the code:

def criar_modelo(hp):
    lstm_input = Input(shape=(x_train_lstm.shape[1], 1), name='LSTM_Input_Layer')
    static_input = Input(shape=(x_train_static.shape[1], ), name='Static_Input_Layer')
    
    # LSTM 1
    
    lstm_layer_1 = LSTM(units=hp.Int('units_lstm_layer_1', min_value=128, max_value=256, step=64), activation='tanh', return_sequences=False, name='1_LSTM_Layer')(lstm_input)
    
    # Static 1 
    
    static_layer_1 = Dense(units=hp.Int('units_static_layer_1', min_value=64, max_value=192, step=64), activation=hp.Choice('activation', ['relu', 'tanh']), name='1_Static_Layer')(static_input)

    # Static 2 e/ou 3
    for i in range(hp.Int('num_static_layers', 1, 3)):
        static_layer = Dense(units=hp.Int(f'static_units_{i}', 128, 192, step=32), activation=hp.Choice('activation', ['relu', 'tanh']), name=f'{i+1}_Static_Layer')(static_layer_1)
        static_layer_1 = static_layer
    
    
    concatenar = Concatenate(axis=1, name='Concatenate')([lstm_layer_1, static_layer_1])
    dense_1 = Dense(units=4*len(np.unique(y_train)), activation='relu', name='1_Dense_Layer')(concatenar)
    dense_2 = Dense(units=2*len(np.unique(y_train)), activation='relu', name='2_Dense_Layer')(dense_1)
    saida = Dense(units=len(np.unique(y_train)), activation='softmax', name='Output_Layer')(dense_2)
    model = Model(inputs=[lstm_input, static_input], outputs=[saida])
    model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
    return model

But, i get the following error:

Traceback (most recent call last):

  Cell In[11], line 27
    tuner = keras_tuner.GridSearch(

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\keras_tuner\src\tuners\gridsearch.py:420 in __init__
    super().__init__(oracle, hypermodel, **kwargs)

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\keras_tuner\src\engine\tuner.py:122 in __init__
    super().__init__(

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\keras_tuner\src\engine\base_tuner.py:132 in __init__
    self._populate_initial_space()

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\keras_tuner\src\engine\base_tuner.py:192 in _populate_initial_space
    self._activate_all_conditions()

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\keras_tuner\src\engine\base_tuner.py:149 in _activate_all_conditions
    self.hypermodel.build(hp)

  Cell In[11], line 23 in criar_modelo
    model = Model(inputs=[lstm_input, static_input], outputs=[saida])

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\tensorflow\python\training\tracking\base.py:629 in _method_wrapper
    result = method(self, *args, **kwargs)

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\keras\engine\functional.py:146 in __init__
    self._init_graph_network(inputs, outputs)

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\tensorflow\python\training\tracking\base.py:629 in _method_wrapper
    result = method(self, *args, **kwargs)

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\keras\engine\functional.py:229 in _init_graph_network
    nodes, nodes_by_depth, layers, _ = _map_graph_network(

  File ~\miniconda3\envs\tf-gpu\lib\site-packages\keras\engine\functional.py:1049 in _map_graph_network
    raise ValueError(

ValueError: The name "1_Static_Layer" is used 2 times in the model. All layer names should be unique.

I don't understand why i get the error, since those i_Static_Layer are defined inside a for loop with range(1, 3).

If i try to print('i =', i), it indeed shows i = 0, but why?

I can always change name=f'{i+1}_Static_Layer' to name=f'{i+2}_Static_Layer', to skip the error. But still, i would like to understand what's going on.

Taha Akbari · Accepted Answer · 2024-03-21 17:57:00Z

1

Note that

hp.Int('num_static_layers', 1, 3)

is first calculated and the output is passed into the range generator. So for example if hp.Int('num_static_layers', 1, 3) return 1 the code will become:

range(1)

which yields 0.

answered Mar 21, 2024 at 17:57

Taha Akbari

2222 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Keras Hyperparameter Tuning with Functional API

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related