Appending and Inspecting JSONL Logs Line by Line in PowerShell – How to Isolate Corrupted Lines

About This Article
This article was generated using an AI-assisted automation workflow. By verifying the formatting requirements of JSON Lines and the PowerShell command specifications in Microsoft Learn, we have compiled code that teaches how to append, read, and handle abnormal lines in dummy logs.

Verification Status: 📘 Official Specifications Confirmed, Hardware Test Pending in PowerShell
The target environment is PowerShell 7 or later. The execution results shown areoutput examplesexpected from the code, not actual hardware execution logs.

When recording processing history in PowerShell, rewriting the entire JSON file every time becomes harder to manage as events increase.

A convenient solution for this isJSON Lines (JSONL, newline-delimited JSON). While standard JSON represents "a single JSON value for the entire file", JSONL placesone independent JSON value per line.

For example, the format is as follows.

{"event_id":"evt-001","status":"START","message":"開始"}
{"event_id":"evt-002","status":"DONE","message":"完了"}

These two lines can be extracted one by one and parsed separately as JSON. Here, we will append two entries using PowerShell, intentionally add an invalid line at the end, and verifyhow far normal reading can be maintained.

What is the difference between JSON and JSONL?

JSONL has three critical formatting requirements.

1. The character encoding must be UTF-8 without BOM. BOM is a special identification byte placed at the beginning.

2. Each line is a valid JSON value by itself. Blank lines are not valid JSON.

3. Separate records using newline characters. LF is standard, and CRLF is also supported.

This isdescribed in the JSON Lines specification. The file extension typically used is .jsonl.

Try it first: Append two dummy events

No administrative privileges or external services are required. Open PowerShell 7 or later and run the following code in anew console. Since it uses only a new temporary directory, existing log files are not modified.

$ErrorActionPreference = 'Stop'

# 毎回異なる一時フォルダーを作り、既存ファイルを触らない

$work = Join-Path ([IO.Path]::GetTempPath()) (
    'jsonl-try-' + [guid]::NewGuid().ToString('N')
)
New-Item -ItemType Directory -Path $work | Out-Null
$file = Join-Path $work 'events.jsonl'

try {
    $events = @(
        [ordered]@{ event_id='evt-001'; status='START'; message='開始' }
        [ordered]@{ event_id='evt-002'; status='DONE';  message='完了' }
    )

    foreach ($event in $events) {

        # -Compressで「1イベント＝1行」にする

        $json = ConvertTo-Json -InputObject $event -Compress -Depth 5
        Add-Content -LiteralPath $file -Value $json -Encoding utf8NoBOM
    }

    # 壊れた3行目を、教材用にだけ追加

    Add-Content -LiteralPath $file -Value '{broken json' -Encoding utf8NoBOM

    $valid = 0
    $invalid = 0
    $lineNo = 0

    foreach ($line in Get-Content -LiteralPath $file -Encoding utf8) {
        $lineNo++
        try {
            $record = ConvertFrom-Json -InputObject $line -ErrorAction Stop
            if ($null -eq $record -or
                [string]::IsNullOrWhiteSpace([string]$record.event_id)) {
                throw 'event_id is missing'
            }
            $valid++
            Write-Output ("[OK] {0} {1}" -f $record.event_id, $record.status)
        }
        catch {
            $invalid++
            Write-Warning ("line {0}: invalid JSON/record" -f $lineNo)
        }
    }

    Write-Output ("[RESULT] valid={0} invalid={1}" -f $valid, $invalid)
}
finally {

    # この実行で作成したフォルダーだけを削除する

    if (Test-Path -LiteralPath $work) {
        Remove-Item -LiteralPath $work -Recurse -Force
    }
}

Expected output example

[OK] evt-001 START
[OK] evt-002 DONE
WARNING: line 3: invalid JSON/record
[RESULT] valid=2 invalid=1

Warning display colors and leading characters vary by host. What you should notice is thateven if the third line is corrupted, the events on the first and second lines are successfully read.

This code does not "fix broken lines." It is an example ofdistinguishingbetween valid and invalid lines. In production systems, you may also need to decide whether to log malformed lines to a separate error record or isolation file rather than silently discarding them.

What does breaking down the code do?

ConvertTo-Json -Compress: fitting into a single line

Converts a PowerShell hashtable into a JSON string.

$data = [ordered]@{
    event_id = 'evt-003'
    status   = 'WARN'
    message  = "1行目" + [Environment]::NewLine + "2行目"
}
$data | ConvertTo-Json -Compress -Depth 5

-Compressremoves visual line breaks and redundant whitespace. Because line breaks within values are escaped in the JSON string,it has the advantage of making it easy to fit one record onto a single lineeven if the message contains line breaks.

-Depthspecifies the depth level to which nested objects are converted. When dealing with deep structures, rather than simply increasing the number, verify whether the required properties remain in the converted JSON.Microsoft Learn's ConvertTo-Jsonserves as the basis for the specification.

Add-Content: appending instead of overwriting

Add-Content -LiteralPath $file -Value $json -Encoding utf8NoBOM

Set-Contentis used to replace existing content, whereasAdd-Contentis used to append to the end.-LiteralPathspecifying prevents symbols in the filename from being interpreted as wildcards.

The reason for explicitly specifying utf8NoBOM in PowerShell 7 is to match the character encoding with the format requirements of JSONL. Because Windows PowerShell 5.1 handles character encodings differently, it is excluded from the scope of this sample as-is. For details on character encoding differences, refer toMicrosoft Learn's Character Encoding Documentation.

Get-Content and ConvertFrom-Json: parsing line by line

foreach ($line in Get-Content -LiteralPath $file -Encoding utf8) {
    $record = ConvertFrom-Json -InputObject $line -ErrorAction Stop
    $record.event_id
}

With a valid file, this allows you to extract each event ID. However,the abbreviated example above stops midway if there is a malformed line, so in practical scenarios, process line by line as shown in the first sample try/catch is used.

PowerShell's Get-Content and ConvertFrom-Json specifications can be reviewed together.

Changing one part to observe the difference in results

Comment out the following single line from the initial code that adds the invalid line.

# Add-Content -LiteralPath $file -Value '{broken json' -Encoding utf8NoBOM

The expected result upon re-execution is [RESULT] valid=2 invalid=0.

What the comparison reveals is that JSONL itself does not automatically repair corrupted data, but rather that the reader implementation limits the blast radius of failures to a single line.

Watch out for empty lines and schema violations

In JSONL, empty lines are also invalid records. However, ConvertFrom-Json may not output anything when passed an empty string. In strict JSONL validation, explicitly filter out blank lines before parsing.

Additionally, even if the JSON is valid, if a mandatory event_id is missing from the event, it is incomplete from a business perspective. While this code minimally checks for the presence of event_id, it does not inspect status allowable values, timestamps, or duplicate event IDs.

For production use: Maintain batch execution history

For example, in automated CSV aggregation processing, the following can be recorded one by one.

{"event_id":"job-042-start","job":"daily-report","status":"START","rows":0}
{"event_id":"job-042-end","job":"daily-report","status":"DONE","rows":145}

Later, you can extract failed processing using event_id or status. The strength is that it serves as both a human-readable log and a format that other tools can ingest sequentially.

However, if multiple PowerShell processes write to a single file simultaneously, atomic writes per line are not guaranteed in this sample. Exclusive control, file locking, deduplication by event ID, and handling of disk full or abrupt termination are required separately. JSONL is a lightweight storage format and is not a substitute for database transactions.

Complete Sample

The code in the main text is a short version for observing behavior. The GitHub educational version includes-KeepFiles, a feature to verify save results, handling of empty and invalid lines, and cleanup routines.

• GitHub Sample — Appending JSONL and validating line by line in PowerShell

The sample isimplemented, but execution verification on a physical PowerShell machine has not been performed. Before targeting production logs in an enterprise environment, please verify the behavior and output using dummy data first.

Official Information and Format Specifications

• JSON Lines — JSONL character encoding, one JSON object per line, and newline conditions

• Microsoft Learn — Compress and Depth parameters in ConvertTo-Json

• Microsoft Learn — Parsing and errors in ConvertFrom-Json

• Microsoft Learn — Appending data and UTF-8 encoding in Add-Content

• Microsoft Learn — Line-by-line reading with Get-Content

• Microsoft Learn — Character encoding and BOM in PowerShell