If you have to stay within the template layer you could use regroup.
{% regroup user_phones|dictsort:"primary" by primary as phones_list %}
{% for phone in phones_list %}
{% if phone.grouper %}
{{ phone.list.0.type }}
{% endif %}
{% endfor %}
What it does
regroup together with the dictsort filter (which also works on querysets) groups the instances in user_phones by the value of primary.
regroup will add an attribute named grouper, which when grouping by a bool (the value of primary) will either be True or False.
for then iterates over the variable phones_list, which is provided by regroup. Since we have sorted the results by primary, {% if phone.grouper %} will tell us when we hit the group of items with primary == True.
regroup packs the items that belong to a group into the attribute list. So the first item can be accessed with phone.list.0.type, phone.list.0.phone_format, etc.
Note:
if you need to access foo.list.0 many times it can be assigned to a variable (using with):
{% regroup user_phones|dictsort:"primary" by primary as phones_list %}
{% for items in phones_list %}
{% if items.grouper %}
{% with items.list.0 as phone %}
<div>{% if phone.type %}{{ phone.type|title }}: {% endif %}<span itemprop="telephone">{{ phone.phone_format }}</span></div>
{% endwith %}
{% endif %}
{% endfor %}