When a script is placed inside angular app element (ng-app on html not body or <script> inside body), <script ng-src="...> will force first bootstrap angular and then only put scripts for download. But mind the fact that if you have multiple libs they will be downloaded in parallel, and order in html won't be tracked in contrary to scripts directly loaded from head at page load.
Alternatively you can add ng-if="doLoadExternalScripts", for example, and specify a truthy value to this variablle when you consider it is right time, for example if your directives are not rendered on init of app