nginx assets not found node js express static rendering

I'm having a similar issue as here, I have a node js express server rendering a build react app ( CRA ) in a folder /public/build. However while using nginx as a reverse proxy, assets get status 404 and only index.html is loaded right.

So I've set up nginx and SSL with Let's Encrypt. I'm deploying a node js express server with ssr (react build) on a VPS in OVH. Let's say my IP is 1.2.3.4 for better reading. If I go to http://1.2.3.4:5000, everything works the same as in localhost, but if I go to https://mysite.fr, then only index.html is here and assets (js, css) get status 404. Please note that I've added an SSL certificate to get HTTPS and I had no issue with it. Here's my nginx config (/etc/nginx/sites-available/default) :

server_name mysite.fr;

location / {
   proxy_pass http://1.2.3.4:5000;
   proxy_http_version 1.1.
   proxy_set_header Upgrade $http_upgrade;
   proxy_set_header Connection 'upgrade';
   proxy_set_header Host $host;
   proxy_cache_bypass $http_upgrade;
   try_files $uri $uri/ =404;
}

SSL certificate was all config by certbot I wrote nothing about it

My architecture :

root/
   -public/
      -build/ ( so my react build app )
         -index.html
   -server.js
   -others folders/

Express config in server.js :

app.use( '/', express.static(path.join(__dirname, 'public/build')));

// Route all other requests to React app index.html
app.get('*', (req, res) => {
    res.sendFile(path.join(__dirname, 'public/build', 'index.html'));
});

So I think that's where I messed up and that my error may come from the way I'm rendering my static files :/ Most stackOverflow answers suggested adding path.join(__dirname but I already had it. I've tried changing port with 80 and then with 443 using this way (server.js) :

//* listen on :80 to get an SSL cert
const server = app.listen(443, function() {
    console.log('Ready on port %d', server.address().port);
});

But it resolved nothing.

When I hadn't added an SSL certificate, http://mysite.fr/static/js/main.f6cd9b6b.js wasn't working either.

I don't know if it's usable, but in index.html : <script defer="defer" src="/static/js/main.f6cd9b6b.js"></script><link href="/static/css/main.fbddab09.css" rel="stylesheet"> links looks like that. In the console of http://1.2.3.4:5000 (STATUS 200), assets request :

Request URL: http://1.2.3.4:5000/static/js/main.f6cd9b6b.js
Request Method: GET
Status Code: 200 OK
Remote Address: 1.2.3.4:5000
Referrer Policy: strict-origin-when-cross-origin

In the console of https://mysite.fr (STATUS 404), assets request :

Request URL: https://mysite.fr/static/css/main.fbddab09.css
Request Method: GET
Status Code: 404 Not Found
Remote Address: 151.80.149.10:443
Referrer Policy: strict-origin-when-cross-origin

Where I'm troubled is that 1.2.3.4 is not working (ok), HTTPS://mysite.fr:5000 not working (ok) but HTTP://mysite.fr:5000 it works fine too. The assets request :

Request URL:http://mysite.fr:5000/static/js/main.f6cd9b6b.js
Request Method:GET
Status Code:200 OK
Remote Address:151.80.149.10:5000
Referrer Policy:strict-origin-when-cross-origin

Here's the log of /var/log/nginx/error.log :

[error] 487919#487919: *372 connect() failed (111:Connection refused) while connecting to upstream, client: XX.XX.XX.XXX, server: mysite.fr, request: "GET / HTTP/1.1", upstream: "http://1.2.3.4:5000", host: "mysite.fr"

Pls note that the XX.XX.XX.XXX Is an IP address I don't know, I think it's mine since it says "client", at least it's not my site IP

I'm confused as to why HTTP://mysite.fr:5000 is working, and I can't find out if the issue is me ( the way I'm rendering static files for example, I really don't know ) or the way I configured nginx. In my head it shouldn't work because it would try to find something like http://1.2.3.4:5000:5000 (*;´□')ゞ I've also noticed that in Content type when it's working it says application/javascript; charset=UTF-8 and when it's not it says text/html.

I'm sorry I'm not able to find out the problem alone, that's my first post I really do need help actually I'm blocked by it for days. (Before nginx I tried using Traefik but only got 404 (even for index ) and the docs was mainly for docker so it was not suited for me. I stopped it with systemctl stop traefik.service.

NB : That's my first time deploying, using node js AND nginx so all of this is really new to me.