JavaFX: Cannot set font size programmatically after font being set by CSS

It's by design. For architectural reasons chain of overrides for css works as follows:

default caspian.css < API settings < user's Scene css < user's Parent css < setStyle()

Here is the quote from css reference guide:

The JavaFX CSS implementation applies the following order of precedence; a style from a user agent style sheet has lower priority than a value set from code, which has lower priority than a Scene or Parent style sheet. Inline styles have highest precedence. Style sheets from a Parent instance are considered to be more specific than those styles from Scene style sheets.

Thus, you can achieve your goal by using setStyle() instead of an API call. Try to run the next example:

public void start(Stage stage) {
    VBox root = new VBox(10);

    Scene scene = new Scene(root, 300, 250);
    // font.css: .labelStyleClass { -fx-font-size: 20 }
    scene.getStylesheets().add(getClass().getResource("font.css").toExternalForm());

    root.getChildren().add(LabelBuilder.create().text("default").build());
    root.getChildren().add(LabelBuilder.create().text("font-css").styleClass("labelStyleClass").build());

    Label lblApi = LabelBuilder.create().text("font-css-api (doesn't work)").styleClass("labelStyleClass").build();
    lblApi.setFont(Font.font(lblApi.getFont().getFamily(), 40));
    root.getChildren().add(lblApi);

    Label lblStyle = LabelBuilder.create().text("font-css-setstyle (work)").styleClass("labelStyleClass").build();
    lblStyle.setStyle("-fx-font-size:40;");
    root.getChildren().add(lblStyle);

    stage.setTitle("Hello World!");
    stage.setScene(scene);
    stage.show();
}