How and when to include shortcode source files

You cannot predict where shortcodes are rendered. Usually in front-end, but there is also AJAX or custom views in back-end.
Include the callback declarations everywhere to be sure.

And if the source file is really that huge, there might be room for refactoring.

If its acceptable, I could suggest you to organize your shortcodes by renaming your all shortcodes to make them having the same name and putting old names into an attribute.

For example, if you have 2 shortcodes [shortcode1 ...] and [shortcode2 ...], then new shortcodes will be [myplugin_shortcode action="shortcode1" ...] and [myplugin_shortcode action="shortcode2" ...].

Your plugin index file:

<?php
/*
Plugin Name: bla bla bla
...
*/

add_shortcode('myplugin_shortcode', 'wpse8170_shortcode_handler');
function wpse8170_shortcode_handler($atts, $content = '') {
    $atts = shortcode_atts(array('action' => false), $atts);
    if (empty($atts['action'])) {
        return '';
    }
    require_once 'shortcodes.php';
    return call_user_func_array("wpse8170_shortcode_{$atts['action']}", array($atts, $content));
}

shortcodes.php:

function wpse8170_shortcode_shortcode1($atts, $content = '') {
    return '...';
}

function wpse8170_shortcode_shortcode2($atts, $content = '') {
    return '...';
}

By organizing your shortcodes like this, you will include php file with shortcode functions only when its really required and it doesn't matter if it is admin or frontend page.