When you use Html5 mode your links became from http://example.com/#about to http://example.com/about but there's one thing why you recieve 404 - what server side solution do you use? As your 404 is returned by your backend rather than by angularJS router itself as in order to make it accessible in html5 mode you need to provide same html page with js included to your new routes on server side just as you've supplied to your home page loading your angular application.
For example in Laravel imagine you have such route for your site root, and your angular app code is in home.blade.php
Route::get('/',function(){ return view('home');});
so you need to create same routes pointing to the same view in order to load your angular js app with markup then your angular router will properly show you your needed template based on routing view.
Route::get('/about',function(){ return view('home');});
Route::get('/contact',function(){ return view('home');});
If your use different server side framework or solution hope you've got main idea of what you need to do to solve 404 problem when accessing directly - just set up same routes on server side pointing to homepage loading your angular js app.
If you dont use any server side framework serving static html file by nginx you need to set up /contacts /about locations to point to your document root index.html which contains your angular App.
for expample
location / {
root /srv/www/example.com/public_html;
index index.html index.htm;
}
location /about/ {
root /srv/www/example.com/public_html;
index index.html index.htm;
}
location /contacts/ {
root /srv/www/example.com/public_html;
index index.html index.htm;
}