Problem when converting DataFrame to DynamicFrame

Question

I have a code that converts DataFrame to DynamicFrame and I get this weird error when trying to execute return statement, any clues what's going on?

Error: {AttributeError}'str' object has no attribute '_jvm'

# record is DynamicFrame
def extractCustomFields(record, ctx):
    rec = record.toDF()
    rec = rec.withColumn("lastname", rec["customfields"][0].value)
    rec.show()
return DynamicFrame.fromDF(rec, ctx, "recordTransform")

fromDF(dataframe, glue_ctx, name) takes GlueContext as second param and you are passing str. Is your GlueContext initialized properly? Please see: fromDf — vladsiv
– vladsiv, Commented Nov 25, 2021 at 11:58

Robert Kossendey · Accepted Answer · 2021-11-25 13:21:05Z

1

fromDF() expects the GlueContext as second argument. You need to pass that:

return DynamicFrame.fromDF(rec, ctx, "recordTransform")

answered Nov 25, 2021 at 13:21

Robert Kossendey

7,1062 gold badges19 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Miroslav Petrovic Over a year ago

yes I was passing transformationContext instead of GlueContext

Collectives™ on Stack Overflow

Problem when converting DataFrame to DynamicFrame

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related