After some contemplating, I came across the vue-cli-service-binary. This binary allows to watch a project and rebuild the app as required. By default, vuecli will output files to dist, which is the directory in which I placed manifest.json, contentScript.js and backgroundScript.js files. The following command will then rebuild the app as needed, allowing for almost hot-reloading (opening and closing the popup). Build times are reasonably quick for a small app (~200ms).
vue-cli-service build --watch --no-clean
Lastly, you might have to disable eslint on save (see this), as it throws errors about chrome not being found. Then I load the unpacked extension into a clean Chrome session and start testing it using the developer tools. Unfortunately, I have not yet gotten Vue Devtools working with the extension.
It works for me: I can develop vuejs-powered Chrome extensions in a reasonabale way. I am still wondering if anybody has any better workflows or ideas for improvement?
EDIT:
@tony19's answer is the preferred way.