How large is the CSS file? I can't think of anything else than splitting it to multiple libraries and then including a library in your page if it's required.
For example, if the page has a slider then it will have a CSS link to slider.css this might not be ideal when you send too many HTTP requests so you might embed the library as internal <style> CSS. I've seen many WordPress theme developers use that technique.
This shouldn't be a problem since minification and Gzipping reduce the filesize drastically and once the file is cached your users wouldn't need to wait extra time to load your next pages.