Am very new to Js world and heard of babel and webpack not sure how to use it
The statement above tells me that setting up npm build pipelines might best be avoided.
So assuming you have the component in jsx and you would like to use the javascript version, you will need to "convert" it. If it is a component that doesn't change quite a lot, you can take a look at the online babel.io repl (this is recommended by the official documentation site as well).
However, this approach can be tedious if your component changes frequently. I can highly recommend create-react-app for development. It is an opinionated toolkit which hides away webpack and babel configuration, but at the same time, their opinions are well documented and work for many general use cases.
Edit
From your comments, it seems you are already using react-scripts, then the most probably problem I see is that you perhaps forgot to specify the homepage property in your package.json (see relevant documentation) By default CRA assumes your static assets are hosted at server root, I assume you are not deploying your WAR in ROOT context, so you need to provide a static location.
I have a similar setup, where I need to package my site built with react inside a war file, I have the following setup:
in package.json
"homepage": "/<webapp_context>/build"
Then with gradle, I copy the build folder in its entirety to the WAR file (same level as WEB-INF).
This instructs react-scripts to put relative paths in all the static assets it publishes (such as CSS, js and images) and the imports then work.