Apply function across pandas dataframe columns

Question

This seems to have been similarly answered, but I can't get it to work.

I have a pandas DataFrame that looks like sig_vars below. This df has a VAF and a Background column. I would like to use the ztest function from statsmodels to assign a p-value to a new p-value column.

The p-value is calculated something like this for each row:

from statsmodels.stats.weightstats import ztest
p_value = ztest(sig_vars.Background,value=sig_vars.VAF)[1]

I have tried something like this, but I can't quite get it to work:

def calc(x):
    return ztest(x.Background, value=x.VAF.astype(float))[1]

sig_vars.dropna().assign(pval = lambda x: calc(x)).head()

It seems strange to me that this works just fine however:

def calc(x):
    return ztest([0.0001,0.0002,0.0001], value=x.VAF.astype(float))[1]

sig_vars.dropna().assign(pval = lambda x: calc(x)).head()

Here is my DataFrame sig_vars:

sig_vars = pd.DataFrame({'AO': {0: 4.0, 1: 16.0, 2: 12.0, 3: 19.0, 4: 2.0},
 'Background': {0: nan,
  1: [0.00018832391713747646, 0.0002114408734430263, 0.000247843759294141],
  2: nan,
  3: [0.00023965141612200435,
   0.00018864365214110544,
   0.00036566589684372596,
   0.0005452562704471102],
  4: [0.00017349063150589867]},
 'Change': {0: 'T>A', 1: 'T>C', 2: 'T>A', 3: 'T>C', 4: 'C>A'},
 'Chrom': {0: 'chr1', 1: 'chr1', 2: 'chr1', 3: 'chr1', 4: 'chr1'},
 'ConvChange': {0: 'T>A', 1: 'T>C', 2: 'T>A', 3: 'T>C', 4: 'C>A'},
 'DP': {0: 16945.0, 1: 16945.0, 2: 16969.0, 3: 16969.0, 4: 16969.0},
 'Downstream': {0: 'NaN', 1: 'NaN', 2: 'NaN', 3: 'NaN', 4: 'NaN'},
 'Gene': {0: 'TIIIa', 1: 'TIIIa', 2: 'TIIIa', 3: 'TIIIa', 4: 'TIIIa'},
 'ID': {0: '86.fastq/onlyProbedRegions.vcf',
  1: '86.fastq/onlyProbedRegions.vcf',
  2: '86.fastq/onlyProbedRegions.vcf',
  3: '86.fastq/onlyProbedRegions.vcf',
  4: '86.fastq/onlyProbedRegions.vcf'},
 'Individual': {0: 1, 1: 1, 2: 1, 3: 1, 4: 1},
 'IntEx': {0: 'TIII', 1: 'TIII', 2: 'TIII', 3: 'TIII', 4: 'TIII'},
 'Loc': {0: 115227854, 1: 115227854, 2: 115227855, 3: 115227855, 4: 115227856},
 'Upstream': {0: 'NaN', 1: 'NaN', 2: 'NaN', 3: 'NaN', 4: 'NaN'},
 'VAF': {0: 0.00023605783416937148,
  1: 0.0009442313366774859,
  2: 0.0007071719017031057,
  3: 0.0011196888443632507,
  4: 0.00011786198361718427},
 'Var': {0: 'A', 1: 'C', 2: 'A', 3: 'C', 4: 'A'},
 'WT': {0: 'T', 1: 'T', 2: 'T', 3: 'T', 4: 'C'}})

Xukrao · Accepted Answer · 2018-10-06 00:54:02Z

1

Try this:

def calc(x):
    return ztest(x['Background'], value=float(x['VAF']))[1]

sig_vars['pval'] = sig_vars.dropna().apply(calc, axis=1)

answered Oct 6, 2018 at 0:54

Xukrao

8,6745 gold badges29 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Apply function across pandas dataframe columns

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related