About this article
This article was created using an automated generation workflow leveraging generative AI. It refers to the official Python json module specification and explains the handling of duplicate keys using fictitious JSON strings.

Verification Status: 📘 Official specification checked, actual execution of the published code is unverified.

What happens if a JSON key appears twice in Python? Testing overrides and duplicate detection

Sometimes it appears as though the first-written value has disappeared after loading a JSON configuration. For example, if a fictitious configuration contains "mode": "test" and "mode": "live" at the same time, which one will be used? When Python's standard library json.loads() converts to a standard dictionary, the last value for the duplicate key remains.It is impossible to recover the fact that there was a duplication using only a regular dictionary.

This behavior does not mean that loading the JSON failed. The issue is that even if the input is ambiguous, the program proceeds by adopting the final value. If you want to handle configuration values or received API data safely, there is a way to check if the same name appeared before conversion to a dictionary.

Target Environment and Scope of Testing

There is no need to directly rewrite actual configuration files or operational data. Start by using only strings and observe how the values change. When incorporating this method into an existing system's configuration, verify not only the receiver's specifications but also the sender's specifications, as well as the contract regarding whether duplicates should be treated as errors.

First, a small experiment: the final value remains

In the following code, the same mode appears twice. The displayed value is the expected result based on the default behavior described in the official Python documentation.

import json

raw = '{"mode":"test","mode":"live","retry":2}'
data = json.loads(raw)
print(data)
print(data["mode"])

The expected dictionary is {'mode': 'live', 'retry': 2} and the value on the second line is live. The first test does not remain because it was overwritten by the same key in the dictionary. It is also important that json.loads() does not throw an exception here. Even if a JSON is syntactically readable, it does not necessarily satisfy the uniqueness expected by the application.

For example, if the retry count is redundantly defined as 2 and 20, it can cause misconfigurations to be overlooked. If only the final value is displayed on the monitoring screen, it becomes harder to investigate where the duplication occurred in the input source. Especially in processes that combine multiple configurations or in steps where templates are edited manually, you should keep in mind the possibility of writing the same key twice.

Why it is difficult to detect after loading

A JSON object is a structure consisting of an ordered list of name/value pairs. In contrast, a standard Python dictionary cannot store two values for the same key simultaneously. If a later value is accepted during conversion, any duplicates present in the original sequence are lost.

The following diagram illustrates the change in information from the original JSON string to the dictionary. Notice how the two occurrences of mode on the left become a single occurrence on the right.

flowchart TD
    A["JSON文字列: mode=test, mode=live"] --> B{"読み込み方法"}
    B -->|通常のloads| C["辞書: mode=live"]
    B -->|object_pairs_hook| D["組の列: mode=test, mode=live"]
    D --> E["重複を検査して拒否できる"]
    C --> F["元の重複は判別しにくい"]

Using the approach on the right side of the diagram, you can receive name/value pairs before they are consolidated into a dictionary. Python's object_pairs_hook is a mechanism for receiving JSON object pairs in order. Rather than simply creating a dict, you can use a custom function to check for duplicates. In this example, files are neither saved nor deleted; instead, encountering invalid input simply raises an exception.

Explicitly halting upon detecting duplicates

The following function checks the received pairs in order and raises ValueError if a key that has already been registered appears. If there are no duplicates through to the end, it returns a dictionary.

import json

def reject_duplicate_keys(pairs):
    result = {}
    for key, value in pairs:
        if key in result:
            raise ValueError(f"duplicate JSON key: {key}")
        result[key] = value
    return result

raw = '{"mode":"test","mode":"live","retry":2}'
try:
    parsed = json.loads(raw, object_pairs_hook=reject_duplicate_keys)
    print(parsed)
except ValueError as exc:
    print(type(exc).__name__, str(exc))

In this example, an error message including duplicate JSON key: mode is expected. At the time the function is called, pairs is a sequence of key/value pairs. Because it stops without completing the dictionary conversion when a duplicate is found, it does not silently accept the later value.

The design decision here is not about "which value takes precedence," but rather "whether to reject ambiguous input." For values related to critical settings or access control, asking the configuration creator to fix the issue is often safer than arbitrarily deciding on first-come-first-served or last-come-first-served rules. However, if existing systems permit duplicates, suddenly introducing this check can impact compatibility, so you should first investigate the actual state of the received data.

Changing a single condition to observe the difference

Let's remove the second mode from the previous string and change it to '{"mode":"test","retry":2}'. Using the exact same check function, a dictionary is now expected to be returned. The only thing changed is the presence or absence of duplicates. Because you can compare normal and abnormal paths without modifying the validation processing code, it becomes easier to understand what the function prohibits.

Let us also consider "mode" and "Mode", where the capitalization of the keys is changed. Because this function compares Python string keys directly, it treats them as distinct names. If business requirements dictate that they should be treated as the same item, you must define separate normalization rules on the application side. However, you must not confuse standard JSON key comparison with business-specific case insensitivity.

Furthermore, what happens with nested objects?object_pairs_hook is called when processing each JSON object, so duplicates are detected even if they occur internally. Meanwhile, having one identical mode in different objects is not a duplication error. The scope of duplication is within each object, not across the entire JSON document.

Troubleshooting Failures: Causes Other Than Duplication

json.loads() itself raises an JSONDecodeError error, first check the JSON syntax, such as quotation marks, commas, and curly braces. For example, trailing commas in JSON are not allowed by the standard Python decoder. A syntax error may occur before reaching the duplicate key detection function. Do not lump syntax errors and duplicate key errors together as simply "broken JSON"; instead, separately record which part of the input needs to be fixed.

object_pairs_hook and object_hook have similar names, but they receive different information. Because the latter receives the object after it has been converted into a standard dictionary, it is not suitable for inspecting original duplicate keys with the same name. Official specifications also state that when both are specified, object_pairs_hook takes precedence. When you want to find duplicates, it is crucial to be aware of the stage at which the function is called.

With massive JSON files or untrusted external inputs, duplication detection alone does not ensure safety. Extremely large data or deep nesting causes processing time and memory consumption issues. Maximum reception size limits, timeouts, allowed items, value types, and schema validation are required separately. The function discussed here is a countermeasure against a single problem—duplicate names—and is not a universal JSON validator.

How to Handle Detection Results in Production

For example, in configuration review tasks, a typical workflow would be to initially store the original JSON, suspend ingestion if a duplication check finds issues, and re-inspect after corrections are made. Rather than investigating errors after overwriting the original input, it is important to be able to trace the content prior to modification. Because outputting the entire JSON to public logs risks leaking secret values, in practice, you should limit this to the minimum necessary information, such as the error type and safe item names.

The same concept applies to JSON received via APIs. However, not all services reject duplicate keys. If the sending and receiving sides interpret things differently, discrepancies can occur with other processes such as signature verification, caching, and auditing. For critical integrations, it is best to explicitly state in the interface specification that "errors will occur if the same key appears multiple times" and prepare both valid and duplicate examples in test data.

If you are only manually checking simple configurations, reviewing and correcting the original string may be sufficient. On the other hand, if you automatically process large volumes of JSON daily, the value of mechanically rejecting duplicates at the ingestion point increases significantly. Whichever you choose, the key takeaway from this experiment is not to conclude that "there were no duplicates in the input" just by looking at the dictionary after loading.

Conclusion

Python's json.loads() normally adopts the last value for duplicate keys. When you want to detect duplicates, use object_pairs_hook, which receives pairs before they become a dictionary. Input uniqueness, syntax correctness, value types, file size, and confidentiality are separate items to verify. Understand the differences using safe, small examples before applying them to your production input contracts.

Official Documentation

• Python Official Documentation: json — JSON encoder and decoder — Reference material confirming the default behavior for duplicate names.object_pairs_hook, reference material confirming decoder specifications.

• RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format — Reference material confirming interoperability notes stating that object names should ideally be unique.