[Pandas]Pandas Dataframe與Json相互轉換

Pandas Dataframe可以用簡單的方式與Json相互轉換,舉一個簡單範例實作 ,用來串接API使用

以下爲簡單的dataframe與Json互轉方法, 若是Spark dataframe可以使用 pd = dataFrame.toPandas() , 轉爲pandas dataframe再進行Json轉換, 但要注意執行toPandas()指令時, 資料不可過大, 以免產生out of memory

# encoding=UTF-8
#!flask/bin/python

from flask import json
import pandas as pd

class JsonConverter(object):

    def sparkDataFrameToJson(self, dataFrame):
        """
        dataframe to Json
        "result": [
        {
          "input": "eee",
          "time": "2017-01-05 10:04:44.054849"
        },
        {
          "input": "gg",
          "time": "2017-01-05 10:17:51.072134"
        },
        {
          "input": "from My Computer",
          "time": "from My Computer"
        }]
        """  
        pd = dataFrame.toPandas()
        return json.loads(pd.to_json(orient='records'))

    def pandasToJson(self, pdf):
        """
        dataframe to Json
        "result": [
        {
          "input": "eee",
          "time": "2017-01-05 10:04:44.054849"
        },
        {
          "input": "gg",
          "time": "2017-01-05 10:17:51.072134"
        },
        {
          "input": "from My Computer",
          "time": "from My Computer"
        }]
        """  

        return json.loads(pdf.to_json(orient='records'))
       
    
    def jsonToSparkDataFrame(self, sqlContext, json):
        """
        Json to dataframe
        {
            "data":[{
                "time": "3",
                "input": "2"
            },
            {
                "time": "5",
                "input": "2"
            }]
        }
        """
        
        pdf = pd.DataFrame(json)

        return sqlContext.createDataFrame(pdf)     

    def jsonToPandas(self, json):
        """
        Json to dataframe
        {
            "data":[{
                "time": "3",
                "input": "2"
            },
            {
                "time": "5",
                "input": "2"
            }]
        }
        """
        
        pdf = pd.DataFrame(json)

        return pdf

若要Json轉dictionary可以使用下面方式

    # JSON到字典转化：
    ret_dict = simplejson.loads(json_str)
    # 字典到JSON转化：
    json_str = simplejson.dumps(dict)