Entity Framework updating entity relation

I am using Entity Framework in my MVC WebApp I can update modified entries but I can't add an relationship.

I know similar questions have been solved here. But I could't find exact answer. I cant update entity

I have an entity cart load.

public class CartLoad
{
public int CartLoadID{get; set;}
public string Description{get; set;}

..... some more properties

public virtual User Driver { get; set; }
}

And a ViewModel which passes data to controller just fine.

public class CartLoadEditView
{
    public CartLoad CartLoad { get; set; }
    public string DriverID { get; set; }
    public SelectList Drivers { get; set; }
}

In but then in controller when I save model all properties of CartLoad are updated except the Driver. Which is and User in table of Users created by Microsoft.Indentity framework.

Controller:

    [HttpPost]
    [ValidateAntiForgeryToken]
    public async Task<ActionResult> Edit(CartLoadEditView CartLoadEdit)
    {
        CartLoad cartLoad = new CartLoad();

        if (ModelState.IsValid)
        {
            User CLDriver = await usrManager.FindByIdAsync(CartLoadEdit.DriverID);
            CartLoadEdit.CartLoad.Driver = CLDriver;

            db.CartLoads.Attach(cartLoad);
            cartLoad = CartLoadEdit.CartLoad;

            db.Entry(cartLoad).State = EntityState.Modified;

            await db.SaveChangesAsync();
            return RedirectToAction("Index");
        }
         ....

When I add breakpoint to

db.Entry(cartLoad).State = EntityState.Modified;

I can see that cartLoad.Driver property is filled with my selected driver but it wont be updated in DB

Solution which I found is to add DriverID property to CartLoad entity and update that. Is that really the right Solution? When EF already added DriverId property to CartLoad entity when using CodeFirst method.

Thanks.