So does it mean that if I use static index.html in which I just
include my react app js file I don't have to use server-side
rendering?
Of course. You can certainly use React purely on the client without any need for server rendering. However server side rendering can be beneficial for graceful degradation. It also helps from a usability perspective as your user won't have to wait for javascript to be downloaded and executed before any content can be shown.
Btw which of react-app architecture better for SEO?
Now search engines have significantly matured in their ability to crawl dynamic pages. However the support for javascript generated content is a work in progress in most engines. As Google Webmasters blog explains:
Sometimes the JavaScript may be too complex or arcane for us to execute, in which case we can’t render the page fully and accurately.
Some JavaScript removes content from the page rather than adding, which prevents us from indexing the content.
So from SEO perspective it is still better if you opt for server side rendering.