A practical checklist
for cleaner records.

1. Profile the source before changing anything

Record the source formats, number of rows, field names, data types, required values, distinct categories and rates of missing values. Identify patterns such as dates in incompatible formats, unexpected characters, repeated identifiers, out-of-range measurements and unusual category values.

Preserve an untouched original or a recoverable copy. The ability to compare before and after is essential to evaluating whether a transformation improved the data.

2. Agree allowable transformations

Define which spellings can be standardised, which categories have approved mappings, how units convert, how dates are interpreted and what counts as a duplicate at the intended record level. Preserve leading zeroes in identifiers and distinguish unknown, not applicable and zero.

For a date written as 03/04/2025, establish the source date convention before interpreting it. For an empty weight, do not invent a plausible value without a valid source. Track the decisions that cannot be resolved from the records alone.

3. Treat duplicates and ambiguous records carefully

Define whether duplicate means identical rows, repeated transactions or multiple records describing the same entity. Similar product names or personal names can lead to false-positive matches. Review candidate groups and specify whether source records should be merged, linked or retained separately.

Document a review method for uncertain cases. Automatic similarity scoring is a triage aid, not independent evidence of identity.

4. Verify outputs against acceptance criteria

Compare counts, key validity, completeness, permitted values, referential consistency and unresolved exceptions before delivering the cleaned dataset. Prepare a clear summary of the rules applied and any fields that could not be verified. Ensure the output format works for the actual receiving application.

Useful deliverables include a prepared export, field definitions, transformation notes, a duplicate-review file and a quality summary. This guide is not a promise that every dataset can become error-free.

To scope an engagement, start with a general description at data cleaning and preparation; sensitive data should only be shared after appropriate arrangements are agreed.