Code splitting not working with webpack 5, react 18, and react router 6

I am trying to set up a new react project with a custom webpack config, and react router. I am using react 18, webpack 5, and react-router 6.

Looking at my App.tsx, it sets up some routes. For now, a few page components are lazy-loaded using React.lazy and Suspense:

import React, { Suspense } from "react";
import { BrowserRouter, Outlet, Route, Routes } from "react-router-dom";

const CarsPage = React.lazy(() => import("pages/cars"));
const PlanesPage = React.lazy(() => import("pages/planes"));


export const App: React.FC = () => {
  return (
    <BrowserRouter>
      <Routes>
        <Route
          element={
            <div>
              <HeaderBar />
              <Outlet />
            </div>
          }
        >
          <Route path="/" element={<div>Homepage here</div>} />
          <Route
            path="/cars/*"
            element={
              <Suspense fallback={<div>Loading . . .</div>}>
                <CarsPage />
              </Suspense>
            }
          />
          <Route
            path="/planes/*"
            element={
              <Suspense fallback={<div>Loading . . .</div>}>
                <PlanesPage />
              </Suspense>
            }
          />
        </Route>
      </Routes>
    </BrowserRouter>
  );
};

I am trying to set up my webpack to codesplit, such that the code for CarsPage or PlanesPage is bundled separately. Following the webpack instructions on codesplitting, this is my webpack config:

module.exports = {
  entry: "./src/index.tsx",
  target: "web",
  output: {
    path: path.resolve(__dirname, "../build"),
    publicPath: "/",
    filename: "[name].js",
    chunkFilename: "[id].[chunkhash].js",
  },
  optimization: {
    splitChunks: {
      chunks: "all",
      cacheGroups: {
        vendors: {
          test: /[\\/]node_modules[\\/]/,
          name: "vendor",
          chunks: "all",
        },
      },
    },
  },
  resolve: {... some resolve rules},
  module: {...some module rules}
  plugins: [...some plugins],
}

When I load the page, I see that the vendor chunk is correctly separate from the main chunk. (This is also desired, as I want my node_modules to be treated as a separate chunkfile, as they'll get updates far less often than the production bundle, so cacheing node_modules in the vendorfile is desired).

Running a build, I am not seeing separate chunk files for the CarsPage and PlanesPage components. Running with webpack dev server, when I load the page, I see a massive main.js file. Navigating to /cars or /planes, no additional js chunk is requested. Clearly, my code is not being split.

I have also tried using one of the new data routers with react-router 6 with the lazy property, similar to what is being done in the question Lazy loading routes in react router v6

What am I doing wrong with my setup here? I know create-react-app used to do this automatically, but how can this be acheived with a custom webpack config? Shouldn't anything imported using the React.lazy import be automatically code-split using webpack 5?