Postgres Data Matching: Find Duplicate and Similar Records Without Exporting a File
Ask anyone who maintains a customer table whether it contains duplicates and you will get the same answer:
almost certainly, but nobody knows how many. The obvious ones were cleaned up years ago. What remains are
the records that are obviously the same to a person and completely different to a database: the account
entered as Acme Corp. in one system and ACME Corporation in another, the customer
who exists twice because someone typed Jim Whitfield the second time instead of
James Whitfield, the shipping address that appears three times with three abbreviations of
the same street.
SQL is excellent at finding exact duplicates and helpless against these. GROUP BY company_name
treats every one of those variations as a distinct company. The usual workaround is to export the table,
run it through a matching tool, and load the results back, which introduces a copy of your production data
that has to be secured, tracked, and deleted, and which is out of date the moment it is written.
The Interzoid Postgres Data Matching Wizard removes that round trip. It connects directly to your PostgreSQL database, reads the columns you choose, and returns clusters of records that refer to the same real-world entity, grouped by an AI-generated similarity key. No export, no staging table, no copy to clean up afterward.
Launch the Postgres Matching Wizard | Get an API Key | Documentation
Why Exact Matching Is Not Enough
Duplicate records are rarely duplicates in the literal sense. They accumulate through mergers, form submissions, imported vendor lists, manual entry, and integrations between systems that each had their own conventions. By the time they land in one table, the same entity may be represented a dozen different ways, none of them wrong, none of them identical. We catalogued how far this can go in One Hundred Ways to Say IBM.
The cost is not aesthetic. It shows up as inflated customer counts, revenue split across records that should have been one account, the same household receiving four copies of a mailing, sales teams calling contacts a colleague already spoke to, and reports that quietly disagree with each other. We have written about the wider financial impact in the hidden cost of poor data quality, and about why it matters even more once that data feeds a model in the critical role of high-quality data in AI success.
Interzoid's approach is described in detail in Simplifying Entity Resolution with Data Matching APIs. Rather than comparing strings, each value is converted into a similarity key that represents the entity behind the text. Values that refer to the same company, person, or place produce the same key regardless of spelling, abbreviation, punctuation, or word order. Grouping by that key turns an intractable comparison problem into an ordinary one.
How the Postgres Matching Wizard Works
The wizard runs in the browser as a guided, five-step workflow. There is nothing to install and no connector to deploy inside your environment.
Choose a Function
Pick from six matching functions covering company names, individual names, addresses, and combinations.
Connect
Enter connection fields or paste a connection string, then pick your database and schema from dropdowns.
Select Columns
Choose the table, assign the match columns, and check the columns to include in the report.
Run
Records are processed concurrently and grouped into clusters by similarity key.
Download
Review the clusters on screen and save the report as a CSV file for review or remediation.
Each step is populated from your live database. After connecting, the wizard queries PostgreSQL for the databases you can access, then the schemas in the one you pick, then the tables and views in that schema, then the columns of the table you select. You never type an object name or look one up in another window, and you cannot select something that is not actually there.
The wizard reads only. It issues SELECT statements against the columns you choose and nothing else. No table is written, no schema object is created, and your database is left exactly as it was found.
Six Ways to Match
Different data calls for different matching. The wizard offers three single-column functions and three combinations that raise precision by requiring agreement on two fields at once.
Company Names
Resolves IBM, I.B.M. Corp, and
International Business Machines to a single entity, along with the legal suffixes,
punctuation, and abbreviations that make company names so variable.
Individual Names
Handles nicknames, initials, and ordering, so James Johnston,
Jim Johnston, and J. Johnston group together instead of counting as
three customers.
Street Addresses
Reconciles directional abbreviations, street type variations, and spacing, matching
400 E Broadway St with 400 East Broadway Street.
Combinations
Company with address, company with individual name, or address with individual name, for cases where one field alone produces matches that are too broad to act on.
Combination functions matter more than they might appear. A large organization may have hundreds of locations sharing one company name, and matching on the name alone collapses them into a single cluster. Requiring the address to agree as well separates the branch in Dallas from the branch in Phoenix while still recognizing that both spell the parent company four different ways.
Connecting to PostgreSQL
The wizard accepts connection details in either of two forms, chosen with a toggle at the top of the connection step.
- Connection fields: host, port, username, password, and SSL mode entered individually.
- Connection string: a complete PostgreSQL connection string in either the URL form,
postgres://user:password@host:5432/database?sslmode=require, or the keyword form,host=myhost user=bob dbname=mydb. The database is taken from the string, so the wizard goes straight to schema selection.
Because a connection string is exactly what managed PostgreSQL providers hand you, the second option usually means pasting one value and connecting. That covers Amazon RDS and Aurora, Azure Database for PostgreSQL, Google Cloud SQL, Supabase, Neon, and any self-managed server reachable over the network. If the platform speaks standard PostgreSQL, the wizard can read from it.
TLS is configurable through the SSL mode setting, which defaults to require so that
cloud-hosted databases are encrypted without any extra step. Certificate validation is available through
verify-ca and verify-full, and disable is there for a local server
that has no TLS configured.
Connect With a Read-Only Role
Since the wizard only reads, the cleanest way to use it is with a role that can only read. A dedicated role takes a moment to create and gives you exact control over what is visible:
CREATE ROLE interzoid_reader LOGIN PASSWORD 'strong-password-here'; GRANT CONNECT ON DATABASE mydb TO interzoid_reader; GRANT USAGE ON SCHEMA public TO interzoid_reader; GRANT SELECT ON ALL TABLES IN SCHEMA public TO interzoid_reader;
Access can be revoked at any time by dropping the role. Credentials are transmitted over HTTPS with each request and are not retained by Interzoid after the session ends. They are saved in your own browser only if you switch on the optional Remember these connection details toggle, which is off by default and erases what it stored when switched back off.
Reading the Results
Results arrive as clusters. Every record that shares a similarity key with at least one other record is shown, grouped together, with a blank line between clusters. The last column of each row is the key itself, which makes the grouping auditable rather than something you have to take on faith.
company,address,city,state,simkey IBM Corporation,1 New Orchard Rd,Armonk,NY,d477E1d7sG6dja3hDNsk9P I.B.M. Corp,1 New Orchard Road,Armonk,NY,d477E1d7sG6dja3hDNsk9P Microsoft Inc.,1 Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T Microsoft Corporation,One Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T MSFT Corp,1 Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T
You choose which columns appear alongside the matched values, so the report can carry the primary keys, account numbers, or region codes your team needs to act on it. Two options control the shape of the output: similarity keys can be suppressed for a cleaner file, and the matches-only filter can be turned off to return every record with its key, which is the form you want when loading results back into a table for further analysis.
The report downloads as a CSV file with no headers or metadata beyond what you asked for, which makes it equally suitable for opening in a spreadsheet, attaching to a ticket, or feeding into a script. A single job can process up to 500,000 records, and each record consumes one API credit.
The interface is available in 17 languages, selectable from the header menu or through a URL parameter
such as postgres-match.interzoid.com/?lang=de.
Where Postgres Matching Pays Off
| Scenario | What Matching Delivers |
|---|---|
| Customer and Account Deduplication | Find the accounts that represent one customer under several spellings, so revenue, activity, and support history can be consolidated against a single record. |
| CRM and Marketing Hygiene | Collapse duplicate contacts before a campaign, so the same person is not emailed three times and engagement metrics reflect people rather than rows. |
| Post-Merger Consolidation | Identify the overlap between two customer or vendor lists loaded into one database, before deciding which records survive consolidation. |
| Vendor and Supplier Rationalization | Reveal how many supplier records refer to the same organization, which is usually the first step in negotiating as one buyer instead of several. |
| Compliance and Risk | Ensure that screening and reporting operate on entities rather than on individual rows, so a match against one spelling is not missed against another. |
| Migration and Modernization | Quantify duplication in a source database before it is carried into a new system, where it becomes considerably more expensive to remove. |
| AI and Analytics Readiness | Give models and dashboards a deduplicated view, so counts, aggregates, and training sets are not distorted by the same entity appearing many times. |
Where Matching Fits in the Workflow
Matching is one step in a sequence, and it works best when the steps around it are in place:
- Before matching, the Data Profiler tells you whether the columns you plan to match on are populated, consistently formatted, and free of placeholder values.
- For files rather than databases, the Match Wizard performs the same matching against CSV and TSV data.
- Snowflake users have a direct equivalent in the Snowflake Data Matching Wizard, which follows the same workflow against a Snowflake warehouse.
- Connecting two datasets whose join values disagree is the job of the Merge Wizard.
- Normalizing company names to their official forms is handled by organization name standardization.
- Filling out sparse records is the role of the Data Enrichment Wizard.
- The broader path from raw data to trustworthy inputs is described in Turn Enterprise Data Into AI-Ready Data.
Every one of these is available through the Interzoid API directory, and the complete set of tools can be launched from Interzoid Central Command.
Frequently Asked Questions
Does the Postgres Data Matching Wizard change my database?
No. The wizard only issues SELECT statements against the table and columns you choose. Nothing is written, updated, or deleted, and no schema objects are created. The match report is returned to your browser, and you decide what to do with it.
Do I have to export my data first?
No. The wizard connects directly to your PostgreSQL database and reads the columns you select. There is no CSV export, no staging area, and no copy of your data to manage or delete afterward.
Which PostgreSQL services are supported?
Any PostgreSQL server reachable over the network, including self-managed instances and managed services such as Amazon RDS and Aurora, Azure Database for PostgreSQL, Google Cloud SQL, Supabase, and Neon. If the platform accepts a standard PostgreSQL connection string, the wizard can read from it.
How does the matching find records that are not identical?
Each value is sent to Interzoid's AI-powered similarity key algorithms, which generate a key representing the underlying entity rather than the literal characters. Two records whose values refer to the same company, person, or address receive the same key, even when the text differs in spelling, abbreviation, punctuation, or word order.
Are my database credentials stored?
Your credentials are transmitted over HTTPS with each connection and discovery request and are not retained by Interzoid after the session ends. They are saved in your own browser only if you switch on the optional Remember these connection details toggle, which is off by default.
What does Postgres data matching cost?
Each record processed consumes one Interzoid API credit, and a single job can process up to 500,000 records. Trial credits are included with a new API account, so a first table can be matched at no cost.
Is the wizard available in other languages?
Yes. The interface is available in 17 languages. Choose a language from the header menu or pass a
language parameter in the URL, for example ?lang=de or ?lang=ja.
Getting Started
A first match takes about ten minutes, most of which is deciding which table to start with.
- Register for an Interzoid API key if you do not already have one; trial credits are included.
- Open the Postgres Data Matching Wizard, enter your API key, and choose a matching function.
- Connect with your connection fields or a connection string, then select your database and schema.
- Choose the table, assign the match columns, and check the columns to include in the report.
- Run the match, review the clusters, and download the CSV report. The documentation covers every option in detail.
A good first table is the one people complain about: the customer list, the contact table, the vendor master. The duplicates are already there, and the only question is how many.
Duplicate records do not announce themselves. They sit in production tables looking like ordinary rows, splitting one customer into four, inflating counts, and quietly undermining every number calculated from them. Exact matching cannot find them, and exporting the table to look for them creates a second problem while solving the first.
The Postgres Data Matching Wizard finds them where they live. Connect to your database, choose a table and a column, and see which of your records have been the same thing all along.
Launch the Matching Wizard | Get an API Key | Read the Documentation | Interzoid Central Command