About this article
This article was created using an automated generation workflow powered by generative AI. Based on Python's official CSV specification and verification records from existing Daily Code samples, it explains the difference between physical line counts and record counts using a hypothetical CSV. This is a self-review from five perspectives (P1 to P5) and is not an independent AI review.

Verification Status: 🧪 12 test records passed for existing Linux/Python 3.13.5 samples. Article publication environment, Windows, and macOS are unverified.

Only 2 records from 4 CSV lines? Distinguishing between "physical lines" and "records" with Python's csv.reader

You open a CSV file and see 4 lines, but the imported data only contains 2 records. This does not necessarily mean data loss or a read failure.This happens because if a cell enclosed in quotes contains a newline, multiple lines in the file are treated as a single record in the CSV.By using Python's standard csv.reader, you can count the number of data records without corrupting cells that contain newlines.

When does this become a problem?

When exporting inquiry lists or survey results to CSV, newlines may be included in fields like "Comments," "Free-form text," or "Work notes." If monitoring staff look only at text editor line numbers or the value of wc -l and conclude that "the imported record count is insufficient," they might misidentify a normal CSV file as an incident. Conversely, treating a CSV with broken quotes as normal is also dangerous.

To understand this distinction, we differentiate among the following three items:

The training material for this section assumes a UTF-8 CSV with exactly one header. Because not all CSVs contain headers, the total number of general CSV records and the CSV_RECORDS displayed here do not have the same meaning.

Let us first examine a fictional CSV

The sample demo.csv contains the following data. Only the memo field for A spans two lines.

id,note
A,"line1
line2"
B,done

Although it appears as four lines in text, it consists of three records (header, A, and B) when interpreted as a CSV. Counting data records only, there are 2 items. The note cell for A contains a line break between line1 and line2. Quotation marks are used to represent values containing commas or line breaks as a single field.

If you simply split strings line by line, you risk misinterpreting the second half of A's memo as a new record. A CSV is not merely a collection of lines separated by commas; it is a format that requires interpreting quotation marks. This is why we do not ingest data using custom split(',').

Preparation and execution steps

No additional packages are required. Python 3.9 or later is assumed. These measurements were taken using Python 3.13.5. The code, tests, dummy CSV, and README are included in the GitHub sample at the end of the article.

1. Download the sample folder andaudit_csv.py、demo.csv、test_audit_csv.py place it in the same folder.

2. Open a terminal and navigate to that folder.

3. Run the following commands one by one.

python3 --version
python3 audit_csv.py demo.csv
python3 -m unittest discover -p 'test_*.py' -v

If on Windows python3 is not recognized, check your Python installation status and try with py -3. There are no measurements yet for running the same command on Windows. Do not confuse PowerShell and Bash commands; verify the version display first.

The CLI output measured on Linux with Python 3.13.5 is as follows.

STATUS=OK
PHYSICAL_LINES=4
CSV_RECORDS=2
HEADER_COLUMNS=2
MULTILINE_RECORDS=1

PHYSICAL_LINES=4 indicates 4 lines in the display, and CSV_RECORDS=2 means 2 entries excluding the header.MULTILINE_RECORDS=1 means there is 1 data record spanning multiple physical lines. A different number does not immediately indicate an error.

How Python makes the distinction

The core of the material is the Python standard library csv.reader. Extracting only the essential parts yields the following.

import csv

with open('demo.csv', encoding='utf-8-sig', newline='') as file:
    reader = csv.reader(file, strict=True)
    header = next(reader)
    records = sum(1 for row in reader)
    print('データ件数:', records)

newline='' is specified to let the CSV parser handle newlines. The official Python documentation also explains specifying this when opening CSV files.utf-8-sig is specified to enable reading files with a UTF-8 BOM. It does not automatically detect character encodings, and there is no guarantee that files such as Shift_JIS can be read correctly.

next(reader) extracts the first CSV record as a header. What is counted in the subsequent iterations is not the number of text newlines, but the number of data records returned by the parser.strict=True is a setting that makes it easier to raise errors for invalid CSV formats, but it does not check whether business field values are correct.

In the official specification, reader.line_num attribute also exists. Judging by the name alone, one might assume it indicates the data record index, but this representsthe physical line position read from the original input. If there are line breaks inside quotation marks, it will not match the record count. In the full version of the training material, compared with the immediately preceding line_num, it counts data records spanning multiple lines.

How the results change when modifying only one condition

Create an alternative CSV with A's notes reformatted into a single line, while leaving the original demo.csv as is.

id,note
A,line1
B,done

The expected value in this case (this modified file itself is not executed this time) is PHYSICAL_LINES=3、CSV_RECORDS=2、MULTILINE_RECORDS=0. The number of data records remains unchanged, and only the physical line count decreases by one. This comparison shows that data increases or decreases cannot be determined solely by changes in physical line counts.

Furthermore, even with CRLF line breaks common on Windows, the training material counts \r\n as a single physical line break. The CRLF case has already been executed in unit tests. However, this is not a program that automatically identifies all CSV files with different character codes or delimiters.

Check the exit code and reason upon failure

The full version is designed not to output CSV contents or actual file paths to standard output, but rather to output only numerical values and error types. The results when testing with a non-existent file are as follows.

In this test, cases including normal, CRLF, BOM, missing trailing newline, empty file, column count mismatch, unclosed quotation mark, invalid UTF-8, missing file, and size exceededall 12 cases succeeded.python3 -m py_compile have also succeeded. The execution was performed in an isolated Linux container and does not represent the result of operating on the Ubuntu production environment or WordPress.

When the file is missing, STATUS=READ_ERROR ISSUE=READ_ERROR is output and confirmed via the CLI to result in an exit code of 3. When integrating into automated processing, check the output for STATUS=OKnot only whether there is,whether the exit code is 0Verify this. If subsequent processes continue even after a failure, the failure to verify the record count itself will be overlooked.

Things to know before bringing this into production

The material used here loads a UTF-8 CSV of 1 MiB or less into memory,read-only validationFor use cases handling massive files, streaming reads or other size control mechanisms are required. Furthermore, even if the column counts match, business quality such as "no duplicate order numbers," "correct dates," and "valid amounts" cannot be proven. Duplicate header names and type checking are separate challenges.

Comparing physical line counts is suitable as theinitial troubleshooting stepwhen consulted about issues like "the imported record count is strange." First, verify the input CSV format, headers, newlines within quotes, and character encoding, and then count the actual CSV records. The natural order is to check exclusion conditions, deduplication, and error row handling on the import destination side afterwards.

There is no need to send actual customer CSVs as-is to AI or external services. If you can reproduce the cause using two fictional items like this time, you can understand the mechanism without using confidential information. Even with a design that does not output file contents to logs, verify log destinations and permissions before using in actual operations.

Conclusion

The "4 lines" and "2 records" in a CSV can coexist because newlines within quoted cells are interpreted bycsv.readeras part of a single record. When checking CSV record counts, distinguish between physical lines and logical records, and explicitly specify how headers are handled.newline='', character encoding specification, and abnormal exit codes makes explanations of import issues more concrete.

Official Documentation and Primary Sources

• Python Official: csv — CSV file reading and writing — csv.reader、newline=''、line_num、strictspecification.

• Python Official: argparse— Command-line option and argument parsing library.

• Python Official: unittest — Running unit tests.

GitHub Sample

• Daily-Code-Samples — Auditing CSV physical rows and record counts