You can do this:
$(function () {
$("img.cat").click(function() {
$("img.cat").css("border","none"); // erases the border on other images
$(this).css('border', "solid 2px #ff0000");
});
});
You just select all the images with the same class again and remove their borders, then continue with setting the border of the one that was just clicked.
Also, as long as you're using jQuery 1.7 (you can use delegate() for earlier versions), then it's recommended that you use on() to attach event handlers. This would look like this:
$(document).on("click", "img.cat", function(){
$("img.cat").css("border","none");
});
To make it more efficient, you could select the closest parent element that all the elements share. For example, if the images were all children of a div with the id imageContainer, you would do this:
$("#imageContainer").on("click", "img.cat", function(){
$("img.cat").css("border","none");
});