Axis limits for scatter plot - Matplotlib

I'm having the same problem presented here, however, the proposed solution didn't work for me.

I'm plotting a set of data which the main plot have this pattern:

Which is a plot which axis limits varies from (-1, 1) in both x and y, with a margin set with this piece of code:

plt.figure()
plt.show(data)
## Add some margin
l, r, b, t = plt.axis()
dx, dy = r-l, t-b
plt.axis([l-0.1*dx, r+0.1*dx, b-0.1*dy, t+0.1*dy])

The problem is 'cause I have more "complex" plot in which some changes had to me made. This is the code that produces it:

def plot_quiver_singularities(min_points, max_points, vector_field_x, vector_field_y, file_path):
    """
    Plot the singularities of vector field
    :param file_path : the path to save the data
    :param vector_field_x : the vector field x component to be plot
    :param vector_field_y : the vector field y component to be plot
    :param min_points : a set (x, y) of min points field
    :param max_points : a set (x, y) of max points  field
    """
    fig = plt.figure(figsize=(8, 8))
    ax = fig.add_axes([.13, .3, .6, .6])
    
    ## Plot quiver
    x, y = numpy.mgrid[-1:1:100*1j, -1:1:100*1j]
    m = numpy.sqrt(numpy.power(vector_field_x, 2) + numpy.power(vector_field_y, 2))
    quiver = ax.quiver(x, y, vector_field_x, vector_field_y, m, zorder=1)
    
    ## Plot critical points
    x = numpy.linspace(-1, 1, x_steps)
    y = numpy.linspace(-1, 1, y_steps)
    
    # Draw the min points
    x_indices = numpy.nonzero(min_points)[0]
    y_indices = numpy.nonzero(min_points)[1]
    ax.scatter(x[x_indices], y[y_indices], marker='$\\circlearrowright$', s=100, zorder=2)
    
    # Draw the max points
    x_indices = numpy.nonzero(max_points)[0]
    y_indices = numpy.nonzero(max_points)[1]
    ax.scatter(x[x_indices], y[y_indices], marker='$\\circlearrowleft$', s=100, zorder=2)
    
    ## Put legends
    marker_min = plt.Line2D((0, 0), (0, 0), markeredgecolor=(1.0, 0.4, 0.0), linestyle='',
                            marker='$\\circlearrowright$', markeredgewidth=1, markersize=10)
    marker_max = plt.Line2D((0, 0), (0, 0), markeredgecolor=(0.2, 0.2, 1.0), linestyle='',
                            marker='$\\circlearrowleft$', markeredgewidth=1, markersize=10)
    plt.legend([marker_min, marker_max], ['CW rot. center', 'CCW rot. center'], numpoints=1,
               loc='center left', bbox_to_anchor=(1, 0.5))
    
    quiver_cax = fig.add_axes([.13, .2, .6, .03])
    fig.colorbar(quiver, orientation='horizontal', cax=quiver_cax)
    
    ## Set axis limits
    plt.xlim(-1, 1)
    plt.ylim(-1, 1)
    
    ## Add some margin
    # l, r, b, t = plt.axis()
    # dx, dy = r-l, t-b
    # plt.axis([l-0.1*dx, r+0.1*dx, b-0.1*dy, t+0.1*dy])
    
    plt.savefig(file_path + '.png', dpi=dpi)
    plt.close()

This produces the following image:

As can be seen, the axis limits do not hold and I didn't found why yet.