You should be thinking of client-side and server-side validation a separate tools to accomplish separate goals.
The reason that you absolutely should be doing validation on the server side is because there is absolutely nothing preventing me from manually editing the Javascript files to remove any client-side validation. You simply can not prevent that, even if your code is minified and obfuscated (both of which can be un-done).
The reason that client-side validation is nice (but not as necessary as server-side validation) is because it actually assists the user in creating a form that will proceed to pass server-side validation. AngularJS can easily validate a form after every keypress. That is awesome from a user perspective. Think of all of those forms where there is a "Password Strength" meter next to the password field, saying that the strength needs to be "good" or better in order to pass validation. If you were to do an ajax validation call after every keypress, you would increase your server load by orders of magnitude.
The idea is that server-side validation is only going to happen after the user hits the "Submit" button. By the time the user even thinks about hitting the "Submit" button, they should already know that their form is going to pass validation.
However, lets get down to your real problem: You want to enter your validation rules only once. That is awesome! Now you are thinking DRY! The only problem is that this would require using something in the back-end to automatically generate your client-side validation, which can be difficult. The alternative could be to have your front end make a call to the server, asking for the rules, and teach your front-end how to interpret the response. This would be an additional call, but it should be a relatively cheap one. Unfortunately, this is quite a bit of extra work. It may be worth it if you expect your form to change frequently in the near future. Being able to change it in one place instead of two would be a huge boon in that situation, and may be worth the additional time spent up-front.
So, is Client side validation absolutely needed? Well, that depends. Customers can be very finicky, so if this is a form that actually generates revenue for you (bank account application form), then you need to realize that some customers might close out of your web-page if they click "submit" and see their form fill up with red all over the place. It is a much better experience if they are told the problems as they happen, rather than all at once.
If, however, you are creating an intranet application on a deadline, then you may need to look at priorities and make a judgment call ;)