Getting error when trying to use ReactJS.Net server side rendering with EPiServer

I am trying to set up server side rendering using ReactJS.Net in an EPiServer project. In order to do that I am using the provided ReactJS.Net Html extention to do my rendering, however, when I run my site

I get an error saying: "ReactJS.NET has not been initialised correctly. Please ensure you have called services.AddReact() and app.UseReact() in your Startup.cs file."

If I swap from trying to use the server rendering extension to trying to render normally with React

I get a different error: "TinyIoCResolutionException: Unable to resolve type: React.Web.BabelHandler".

I think the root issue is that there is some difference in how an EPiServer project initializes things at start up vs a vanilla .Net MVC project, but I am unsure of what the difference is or how to work around it.

I am working with the React.Web.Mvc4 v4.0.0 nuget package, Episerver 11, and .Net Framework 4.6.2. I took the same component I'm trying to render and pasted it into a vanilla .Net MVC project with no EPiServer and it renders properly client side and server side using the extension, so the component is not the issue. I found something saying that this error is usually seen when React or ReactDOM is not exposed to the global scope so I set it up to ensure that my react component is being added to the global scope, but no luck there either.

Here is my ReactConfig:

public static void Configure()
{
    JsEngineSwitcher.Current.DefaultEngineName = V8JsEngine.EngineName;
    JsEngineSwitcher.Current.EngineFactories.AddV8();

    ReactSiteConfiguration.Configuration
        .AddScript("~/Static/js/Components/PrimaryCallout.jsx");
}

Here is my component:

class PrimaryCallout extends React.Component {
    constructor(props) {
        super(props);
        this.state = {
            header: this.props.header,
            buttonText: this.props.buttonText,
            buttonLink: this.props.buttonLink
        };
    }

    render() {
        return (
            <div className="primary-callout-container">
                <h2 className="primary-callout-header">{this.state.header}</h2>
                <a href={this.state.buttonLink} className="primary-callout-link">
                    <button className="primary-callout-button">{this.state.buttonText}</button>
                </a>
            </div>
        );
    }
}

Here is how I'm trying to render my component:

@Html.React("PrimaryCallout", new
{
    header = Model.Header,
    buttonText = Model.CalloutLinkText,
    buttonLink = Url.ContentUrl(Model.CalloutLink)
})

Any and all help or insight is appreciated.

Thank you.