I don't think this can be accomplished with the ContentSizeFitter and LayoutElement components. Instead, I think you will need a custom script that sniffs the device's screen size and computes the aspect ratio and adjusts the RectTransform accordingly.

Here is a demo to show how this can be done. For this example we have a canvas with a panel child. The panel will be the target of our scaling.

The canvas' CanvasScaler component can be set to either constant pixel size or scale with screen size -- both work. The panel's RectTransform anchor preset should be set to stretch/stretch and the top, bottom, left and right values should all be set to zero to have the panel fill the canvas by default.

For demo purposes, the script is attached to an empty gameobject. Also for demo purposes, the scaling routine runs in the Update loop -- something you would obviously never want to do in a real application. Here's the script:

using UnityEngine;

public class AspectRatioDemo : MonoBehaviour
{
    public RectTransform panel;
    public float maxAspectRatio = 1080f / 1920f; // example: Nexus 6P

    // routine shown in Update for demo purposes, obviously you would only do this once...
    void Update()
    {
        float currentAspectRatio = (float)Screen.width / (float)Screen.height;
        if(currentAspectRatio > maxAspectRatio)
        {
            float padding = (currentAspectRatio - maxAspectRatio)/2f;
            if(padding > .5f)
            {
                Debug.LogError("unsupported aspect ratio"); //addition logic required to support landscape mode
                return;
            }
            panel.anchorMax = new Vector2(1.0f - padding, 1.0f);
            panel.anchorMin = new Vector2(padding, 0);
        }
        else // this is only necessary if panel was previously resized
        {
            panel.anchorMax = Vector2.one;
            panel.anchorMin = Vector2.zero;
        }
    }
}

maxAspectRatio represents the widest aspect you'll allow before padding the panel's margins. By default I have it set to 1080/1920 (.5625). The routine checks whether the actual aspect ratio exceeds this by dividing Screen.width by Screen.Height. If not, it does nothing. If it does, it computes a padding value. (Note the if the computed padding value is greater than .5 we're talking about a device in landcape mode which is unexpected and unsupported here.) Finally the code adjusts the anchors to create the padding.