I have 5V -> button -> 10K resistor -> PIN2 setup with Arduino UNO.

My code is very simple:

int count = 0;

void setup() {
  pinMode(2, INPUT);
  Serial.begin(9600); 
}

void loop() {
  int state = digitalRead(2);
  Serial.println(state);
  if (state == HIGH) {
    count++;
    Serial.print("Hi");
    Serial.println(count);
  }
  delay(1000); 
}

If the button is pressed print "HiX", but what I get is very strange. When I press the button only once I get this:

0
0
0
0
0
1
Hi1
1
Hi2
1
Hi3
1
Hi4
1
Hi5
1
Hi6
1
Hi7
1
Hi8
1
Hi9
1
Hi10
0

The button press is registered 10 times.

What am I doing wrong ?