Custom JSON Decoder in Python

JSON objects are one of the most efficient tools to communicate with web applications. When we receive a JSON file, we need to convert it into a python object to use it in our python program. In this article, we will discuss different ways to create and use a custom JSON decoder in Python.

Before continuing with the article, I suggest you read this article on working with JSON files in Python if you don’t know how to work with simple JSON objects.

How to Convert JSON to Python object?

You can convert a JSON file or string to a python object using the load() method, the loads() method, or the JSONDecoder class. Let us discuss each method one by one.

JSON File to Python Object Using the load() Method

The load() method takes a file pointer to a JSON file and returns a python dictionary object. For instance, we have the following JSON file.

When we convert this file to a python object using the load() method, we get a python dictionary as shown in the following example.

import json
fp=open("simplestudent.json")
myDict=json.load(fp)
print("The python object is:")
print(myDict)

Output

The python object is:
{'Name': 'Aditya', 'Age': 23}

If you want to get a python object instead of the dictionary, we need to create a custom JSON decoder. For this, we will create a function that takes the dictionary returned by the load() method and converts it into a python object. We will pass the function to the object_hook parameter in the load() method while encoding the JSON file. You can observe this in the following example.

import json
class Student:
    def __init__(self, name, age):
        self.Name=name
        self.Age=age
def SimpleDecoderFunction(jsonDict):
    return Student(jsonDict["Name"],jsonDict["Age"])
fp=open("simplestudent.json","r")
python_obj=json.load(fp,object_hook=SimpleDecoderFunction)
print("The python object is:")
print(python_obj)

Output:

The python object is:
<__main__.Student object at 0x7fe1c87a36a0>

In the above example, we have defined a Student class. We have also defined a SimpleDecoderFunction() function. When we pass the SimpleDecoderFunction() to the load() method while decoding a JSON object, the created python dictionary object is first sent to the SimpleDecoderFunction(). The SimpleDecoderFunction() takes the dictionary and converts it into a python object of the Student class that we get as the output of the load() method.

Json String to Python Object Using loads() Method

If you have a JSON string instead of a JSON file, you can convert it into a python object using the loads() method. The loads() method takes a JSON string as its input argument and returns a python dictionary as shown in the following example.

import json
jsonStr='{"Name": "Aditya", "Age": 23}'
python_obj=json.loads(jsonStr)
print("The python object is:")
print(python_obj)

Output:

The python object is:
{'Name': 'Aditya', 'Age': 23}

To convert a JSON string to a python object using the loads() method, you can use a custom JSON decoder function and the object_hook parameter as shown below.

import json
class Student:
    def __init__(self, name, age):
        self.Name=name
        self.Age=age
def SimpleDecoderFunction(jsonDict):
    return Student(jsonDict["Name"],jsonDict["Age"])
jsonStr='{"Name": "Aditya", "Age": 23}'
python_obj=json.loads(jsonStr,object_hook=SimpleDecoderFunction)
print("The python object is:")
print(python_obj)

Output:

The python object is:
<__main__.Student object at 0x7fe1c87a17b0>

You can observe that the loads() method works in a similar manner to the load() method. The only difference is that it reads the JSON object from a string instead of a file.

Instead of using the load() method and the loads() method, we can also create a decoder using the JSONDecoder class to convert JSON objects into python objects.

Suggested Reading: If you are into machine learning, you can read this article on mlops for beginners. You might also like this article on clustering mixed data types in Python.

JSON File to Python Object Using JSONDecoder Class

The JSONDecoder class constructor has the following syntax

class json.JSONDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)

Here,

• The object_hook parameter is used to create custom JSON decoders. The object_hook parameter takes a function as its input argument. The function is called with the object literal decoded from the JSON. In the output, the return value of the function is used instead of the dict. 
• The parse_float parameter is used to convert any floating point number in the JSON to another data type. By default, the float() function is called with the strings containing floating point numbers in the JSON while decoding. If we specify a function in the parse_float parameter,  the decoder passes the string containing a floating point number to the function and the output of the function is used in the python object. This parameter can be used if you want to convert the floats to ints or other data types while loading the JSON itself.
• The parse_int parameter is used to convert any integer in the JSON to another data type. By default, the int() function is called with the strings containing integers in the JSON. If we specify a function in the parse_int parameter,  the decoder passes the string containing the integer to the function and the output of the function is used in the python object. This parameter can be used if you want to convert the integers to floats or other data types while loading the JSON itself. The default parse_int of int() now limits the maximum length of the integer string via the interpreter’s integer string conversion length limitation to help avoid denial of service attacks.
• The parse_constant parameter is used to load NaN, -Infinity, and +Infinity from JSON to custom python values. The parse_constant parameter takes a function as its input argument. While the execution of the decoder,  NaN, -Infinity, and +Infinity are passed to the function, and the return value is used in the python object.
• The object_pairs_hook is an optional parameter that takes a function as its input argument. The function is called with the result of any object literal decoded with an ordered list of pairs. The return value of object_pairs_hook is used instead of the dict. This feature can be used to implement custom decoders. If object_hook is also defined, the object_pairs_hook takes priority.

After execution, the JSONDecoder() constructor returns a JSON decoder. We can invoke the decode() method on the JSON decoder to obtain a python dictionary from a JSON string as shown below.

import json
jsonStr='{"Name": "Aditya", "Age": 23}'
print("The JSON string is:")
print(jsonStr)
myDict=json.JSONDecoder().decode(jsonStr)
print("The python object is:")
print(myDict)

Output:

The JSON string is:
{"Name": "Aditya", "Age": 23}
The python object is:
{'Name': 'Aditya', 'Age': 23}

In the above example, we first create a JSONDecoder object using the JSONDecoder() constructor. After that, we invoke the decode() method on the JSONDecoder object. The decode() object takes a JSON string as its input argument and returns a Python dictionary.

To convert a JSON string to a custom python object, you can use the object_hook parameter in the JSONDecoder() constructor. The JSONDecoder() constructor takes a function as its input argument. The function must take the dictionary which is the normal output while decoding and convert it to a custom python object. For instance, consider the following example.

import json
class Student:
    def __init__(self, name, age):
        self.Name=name
        self.Age=age
def SimpleDecoderFunction(jsonDict):
    return Student(jsonDict["Name"],jsonDict["Age"])
jsonStr='{"Name": "Aditya", "Age": 23}'
python_obj=json.JSONDecoder(object_hook=SimpleDecoderFunction).decode(jsonStr)
print("The python object is:")
print(python_obj)

Output:

The python object is:
<__main__.Student object at 0x7fe1c87a32b0>

Nested JSON File to Python Object Using Custom Decoder Class

Converting a flat JSON file to a python object is easy as all the values in the JSON object are of primitive data types when converted to a dictionary. However, decoding nested JSON objects gives us nested dictionaries.  

If we convert the above string to JSON, we will get a nested dictionary as shown below.

import json
jsonStr='{"__type__": "Student","Name": "Aditya", "Age": 23, "Details": {"__type__": "Details","Height": 160, "Weight": 60}}'
python_obj=json.JSONDecoder().decode(jsonStr)
print("The python object is:")
print(python_obj)

Output:

The python object is:
{'__type__': 'Student', 'Name': 'Aditya', 'Age': 23, 'Details': {'__type__': 'Details', 'Height': 160, 'Weight': 60}}

For converting nested JSON files to python objects, there should be a key-value pair in the JSON that determines the type of python object that we want to create. If the JSON object contains the type of the python object to be created, we can define a custom function that takes the dictionary loaded from the JSON object and converts it into a python object. Then, we will pass the function to the object_hook parameter in the load() method. After this, the load() method will return a custom python object instead of a dictionary. You can observe this in the following example.

class Student:
    def __init__(self, name, age,details):
        self.Name=name
        self.Age=age
        self.Details=details
class Details:
    def __init__(self, height, weight):
        self.Height=height
        self.Weight=weight
def ComplexDecoderFunction(jsonDict):
    if '__type__' in jsonDict and jsonDict['__type__'] == 'Student':
        return Student(jsonDict['Name'], jsonDict['Age'], jsonDict['Details'])
    if '__type__' in jsonDict and jsonDict['__type__'] == 'Details':
        return Details(jsonDict['Height'], jsonDict['Weight'])
    
fp=open("complexstudent.json")
python_obj=json.load(fp,object_hook=ComplexDecoderFunction)
print("The python object is:")
print(python_obj)
fp.close()

Output:

The python object is:
<__main__.Student object at 0x7fe1c87a2d70>

In the above example, we have defined a Details class with attributes Height and Weight. We have also defined the Student class with the attributes Name, Age, and Details.

To convert the input nested dictionary to a python object, we have defined a custom JSON decoder function ComplexDecoderFunction(). The input json object has the attribute __type__ to specify the class of python object to which the object can be converted. The process of encoding complex python objects to JSON is discussed in this article on custom json encoder in Python.

The load() method passes the outer dictionary and the inner dictionary to the ComplexDecoderFunction(). The function checks the class to which the dictionary has to be converted using the __type__ attribute and returns a python object of the appropriate type. The load() method then returns the complete python object.

If you want to obtain a python object from a json string instead of a file, you can use the loads() method instead of the load() method as shown in the following example.

class Student:
    def __init__(self, name, age,details):
        self.Name=name
        self.Age=age
        self.Details=details
class Details:
    def __init__(self, height, weight):
        self.Height=height
        self.Weight=weight
def ComplexDecoderFunction(jsonDict):
    if '__type__' in jsonDict and jsonDict['__type__'] == 'Student':
        return Student(jsonDict['Name'], jsonDict['Age'], jsonDict['Details'])
    if '__type__' in jsonDict and jsonDict['__type__'] == 'Details':
        return Details(jsonDict['Height'], jsonDict['Weight'])
    
jsonStr='{"__type__": "Student","Name": "Aditya", "Age": 23, "Details": {"__type__": "Details","Height": 160, "Weight": 60}}'
python_obj=json.loads(jsonStr,object_hook=ComplexDecoderFunction)
print("The python object is:")
print(python_obj)

Output:

The python object is:
<__main__.Student object at 0x7fe1c87a1f90>

You can also create a custom decoder for nested json strings to create a python object using the JSONDecoder() constructor as shown below.

class Student:
    def __init__(self, name, age,details):
        self.Name=name
        self.Age=age
        self.Details=details
class Details:
    def __init__(self, height, weight):
        self.Height=height
        self.Weight=weight
def ComplexDecoderFunction(jsonDict):
    if '__type__' in jsonDict and jsonDict['__type__'] == 'Student':
        return Student(jsonDict['Name'], jsonDict['Age'], jsonDict['Details'])
    if '__type__' in jsonDict and jsonDict['__type__'] == 'Details':
        return Details(jsonDict['Height'], jsonDict['Weight'])
    
jsonStr='{"__type__": "Student","Name": "Aditya", "Age": 23, "Details": {"__type__": "Details","Height": 160, "Weight": 60}}'
python_obj=json.JSONDecoder(object_hook=ComplexDecoderFunction).decode(jsonStr)
print("The python object is:")
print(python_obj)

Output:

The python object is:
<__main__.Student object at 0x7fe1c87a31f0>

Conclusion

In this article, we have discussed different ways to create a custom json decoder in python. To learn more about python programming, you can read this article on how to create a chat app in Python. You might also like this article on linear regression using the sklearn module in Python.