You're getting noise pickup on the input pin when the button is not pressed. You need a pulldown resistor, about 10k to ground, to quiet the noise when the switch is open.
Or even easier, use the built-in pullup resistor in the Arduino, which is ~10K to +5v, wire your button to ground instead of +5v, and reverse the if tests to test for logic LOW instead of HIGH, to detect a button press. You'd change your pinmode statement to:

pinMode(switchPin, INPUT_PULLUP);  // engage the input pullup

and your test to:

 if (digitalRead(switchPin) == LOW ){ // Negative logic

No additional (external) hardware needed!

You're getting noise pickup on the input pin when the button is not pressed. You need a pull-down resistor, about 10 kΩ to ground, to quiet the noise when the switch is open.
Or even easier, use the built-in pull-up resistor in the Arduino, which is ~10 kΩ to +5 V, wire your button to ground instead of +5 V, and reverse the if tests to test for logic LOW instead of HIGH, to detect a button press. You'd change your pinmode statement to:

pinMode(switchPin, INPUT_PULLUP);  // engage the input pullup

and your test to:

 if (digitalRead(switchPin) == LOW ){ // Negative logic

No additional (external) hardware is needed!