webpack can't build typescript react

I'm trying to build a typescript react project with webpack. I followed the typescript Tutorial, but keep getting the error `module parse failed: ... you may need an appropriate loader``

I can compile using tsc so the problem isn't in typescript itself.

The failure occurs at the <div> in the following snippet

ReactDOM.render(
    <div>Something</div>,
    document.getElementById("content")
);

If I replace the <div> with null, webpack compiles.

I'm not sure what I'm missing here, I'm guessing it's something with how jsx/tsx is loaded?

My webpack.config is

module.exports = {
    entry: "./src/components/index",
    output: {
        filename: "bundle.js",
        path: __dirname + "/dist"
    },

    // Enable sourcemaps for debugging webpack's output.
    devtool: "source-map",

    resolve: {
        // Add `.ts` and `.tsx` as a resolvable extension.
        extensions: ['.ts', '.tsx', '.js']
      },
      module: {
        rules: [
          // all files with a `.ts` or `.tsx` extension will be handled by `ts-loader`
          {
            test: /\.tsx?$/,
            use: 'awesome-typescript-loader',
          }
        ]
      },
      externals: {
        'react': 'React',
        'react-dom': 'ReactDOM',
      },
};

----------- update ----------------

React and React-Dom are included in the file index.tsx as

import * as React from "react";
import * as ReactDOM from "react-dom";

JSX is enabled in the tsconfig file as "jsx": "react"

Also, .babelrc includes

{
  "plugins": ["transform-runtime","array-includes"],
  "presets" : ["env", "react", "stage-2"]
}

----------- update 2 -------------- I suspect this is an issue with webpack loader not loading propertly. If I change the loader from

module: {
        rules: [
            // All files with a '.ts' or '.tsx' extension will be handled by 'awesome-typescript-loader'.
            { test: /\.tsx?$/, loader: "awesome-typescript-loader" },

to

module: {
        rules: [
            // All files with a '.ts' or '.tsx' extension will be handled by 'awesome-typescript-loader'.
            { test: /\.tsx?$/, loader: "some-random-loader-name" },

I would expect it to error that the loader can't be found, but I still get the same behaviour as I did when using awesome-typescript-loader.