0

The schema of data that read from hdfs is:

root
|-- id: string
|-- ext_json: string

while the data in ext_json is like:

[{'a':'1','b':'2'},{'a':'3','b':'4'}]

now I need to convert the data that schema is as follows:

root
|-- id: string
|-- ext_json: array
|    |-- element: struct
|    |    |-- a: string
|    |    |-- b: string

How to do that?

Spark version is 2.0.1

Improve this question
2
  • hope this helps stackoverflow.com/a/42069295/1025328 Commented Feb 7, 2017 at 10:31
  • @PrasadKhode yeah it's the same problem, but I got error: data type mismatch: input to function explode should be array or map type, not StringType Commented Feb 7, 2017 at 11:42
Related questions
3881
Convert bytes to a string in Python 3
7403
How to check whether a string contains a substring in JavaScript?
4771
How do I read / convert an InputStream into a String in Java?
Load 7 more related questions Show fewer related questions

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.