Implementing convolutional layers using Tensorflow

Question

I am trying to implement convolutional layers for text classification from this blog post with some modification to suit my needs.

In the blog, there is only one convolution layer while I'd like mine to have two convolutional layers followed by ReLU and max-pooling.

The code so far is:

vocab_size = 2000
embedding_size = 100
filter_height = 5
filter_width = embedding_size
no_of_channels = 1
no_of_filters = 256
sequence_length = 50
filter_size = 3
no_of_classes = 26


input_x = tf.placeholder(tf.int32, [None, sequence_length], name="input_x")
input_y = tf.placeholder(tf.float32, [None, no_of_classes], name="input_y")



# Defining the embedding layer:

with tf.device('/cpu:0'), tf.name_scope("embedding"):
    W = tf.Variable(tf.random_uniform([vocab_size, embedding_size], -1.0, 1.0), name="W")
    embedded_chars = tf.nn.embedding_lookup(W, input_x)
    embedded_chars_expanded = tf.expand_dims(embedded_chars, -1)


# Convolution block:

with tf.name_scope("convolution-block"):
    filter_shape = [filter_height, embedding_size, no_of_channels, no_of_filters]
    W = tf.Variable(tf.truncated_normal(filter_shape, stddev=0.1), name="W")
    b = tf.Variable(tf.constant(0.1, shape=[no_of_filters]), name="b")

    conv1 = tf.nn.conv2d(embedded_chars_expanded,
                   W,
                   strides = [1,1,1,1],
                   padding = "VALID",
                   name = "conv1")

    conv2 = tf.nn.conv2d(conv1,
                    W,
                    strides = [1,1,1,1],
                    padding = "VALID",
                    name = "conv2")

Here, W is the filter matrix.

However, this gives the error:

ValueError: Dimensions must be equal, but are 256 and 1 for 'convolution-block_16/conv2' (op: 'Conv2D') with input shapes: [?,46,1,256], [5,100,1,256].

I realise I have erred in the dimensions of the layer, but I am unable to fix it or put in the correct dimensions.

If anybody could provide any guidance/help, it'd be really helpful.

Thank you.

Tianjin Gu · Accepted Answer · 2017-09-02 11:23:11Z

1

Can't quite understand what you code to do, but change as follows will fix your problem.

with tf.name_scope("convolution-block"):
    filter_shape = [filter_height, embedding_size, no_of_channels, no_of_channels #change the output channel as input#]
    W = tf.Variable(tf.truncated_normal(filter_shape, stddev=0.1), name="W")
    b = tf.Variable(tf.constant(0.1, shape=[no_of_filters]), name="b")

    conv1 = tf.nn.conv2d(embedded_chars_expanded,
                   W,
                   strides = [1,1,1,1],
                   padding = "SAME", ##Change the padding scheme
                   name = "conv1")

    conv2 = tf.nn.conv2d(conv1,
                    W,
                    strides = [1,1,1,1],
                    padding = "VALID",
                    name = "conv2")

answered Sep 2, 2017 at 11:23

Tianjin Gu

7946 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Implementing convolutional layers using Tensorflow

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related