applying a lambda function to pandas dataframe

The default settings for pandas functions typically used to import text data like this (pd.read_table() etc) will interpret the spaces in the first 2 column names as separators, so you'll end up with 6 columns instead of 4, and your data will be misaligned:

In [23]: df = pd.read_clipboard()

In [24]: df
Out[24]:
   start        lat    start.1       long    end_lat  end_long
0      0  38.902760 -77.038630  38.880300 -76.986200       NaN
1      2  38.895914 -77.026064  38.915400 -77.044600       NaN
2      3  38.888251 -77.049426  38.895914 -77.026064       NaN
3      4  38.892300 -77.043600  38.888251 -77.049426       NaN

In [25]: df.columns
Out[25]: Index(['start', 'lat', 'start.1', 'long', 'end_lat', 'end_long'], dtype='object')

Notice column names are wrong, the last column is full of NaNs, etc. If I apply your function to the dataframe in this form, I get the same error as you did.

Its usually better to try to fix this before it gets imported as a dataframe. I can think of 2 methods:

1. clean the data before importing, for example copy it into an editor and replace the offending spaces with underscores. This is the easiest.
2. use a regex to fix it during import. This may be necessary if the dataset is very large, or its is pulled from a website and has to be refreshed regularly.

Here's an example of case (2):

In [35]: df = pd.read_clipboard(sep=r'\s{2,}|\s(?=-)', engine='python')

In [36]: df = df.rename_axis({'start lat': 'start_lat', 'start long': 'start_long'}, axis=1)

In [37]: df
Out[37]:
   start_lat  start_long    end_lat   end_long
0  38.902760  -77.038630  38.880300 -76.986200
2  38.895914  -77.026064  38.915400 -77.044600
3  38.888251  -77.049426  38.895914 -77.026064
4  38.892300  -77.043600  38.888251 -77.049426

The specified that separators must contain either 2+ whitespaces characters, or 1 whitespace followed by a hyphen (minus sign). Then I rename the columns to what i assume are the expected values.

From this point your function / apply works fine, but i've changed it a little:

• PEP8 recommends putting imports at the top of each file, rather than in a function
• Extracting the columns by name is more robust, and would have given a much more understandable error than the weird error thrown by geopy.

For example:

In [51]: def dist_calc(row):
    ...:    start = row[['start_lat','start_long']]
    ...:    end = row[['end_lat', 'end_long']]
    ...:    return vincenty(start, end).miles
    ...:

In [52]: df.apply(lambda row: dist_calc(row), axis=1)
Out[52]:
0    3.223232
2    1.674780
3    1.365851
4    0.420305
dtype: float64