预处理记录相似度API是一个为现实世界的杂乱数据构建的高速模糊匹配和去重API 它帮助你识别近重复记录并调解实体即使值不完全匹配—拼写错误、大写差异、缺少标点、间距问题、缩写和轻微的词序变化
你无需构建和调整自己的模糊匹配管道,只需将你的字符串(或记录)发送到API并返回可以信任的相似度评分匹配典型输出包括匹配对(例如:“苹果” ↔ “苹果公司”)、相似度分数和结构化结果,便于插入数据清理工作流、客户关系管理、ETL作业和分析管道
常见用例:
去重列表:在数据集中查找重复项(全对全匹配)并返回可能的重复对
与主列表调解:将输入列表与标准集匹配(列表对主表)
客户关系管理和客户数据卫生:清理导致报告和外联出现问题的潜在重复的线索/账户/公司
实体解析与记录链接:在多个来源中连接到同一现实世界实体的引用
团队使用它的原因:
开箱即用处理杂乱文本(不需要为每个边缘案例手动规则)
用于排名和阈值的相似度分数(你可以选择多严格)
为规模和自动化而构建(设计用于在管道中运行,而不仅仅是一次性脚本)
Dedupe is an all-to-all fuzzy matching endpoint for finding duplicates within a single list of strings. Instead of comparing only two inputs per API call, you send a dataset and it returns similar pairs and/or deduplicated groups across the entire set.
Why you’d use it
Massive speedup: typically ~300× to 1,000× faster than “regular” approaches people try first (pairwise comparisons, looping fuzzy scorers, etc.) once you go beyond tiny lists.
Optional cleanup built-in: you can enable common text cleanup (lowercasing, punctuation removal, token sorting). This saves hours (or days) of development + ongoing maintenance.
Company suffixes handled automatically: common endings like “Inc”, “LLC”, “Ltd”, etc. are stripped so you match the real name.
Benchmarks: similarity-api/blog/speed-benchmarks (1M records in ~7 minutes; faster than common Python fuzzy matching libraries).
Hard limits on Zyla
Max 1,000 strings per request (enforced).
Need bigger / unlimited?
Parameters (POST request)
data (required)
A string containing a JSON array of strings.
Example value for data:
["Acme Inc","ACME LLC","Globex GmbH"]
Higher = stricter matching (fewer pairs). Typical: 0.80–0.90 for company dedupe.
Removes punctuation differences (e.g., “A.C.M.E.” vs “ACME”).
Makes matching case-insensitive.
use_token_sort (optional, true/false, default false)
Helps when word order changes (e.g., “Bank of America” vs “America Bank of”).
output_format (optional, default string_pairs)
This exndpoint can return data in multiple formats. Please select one of the following:
string_pairs:
[string_A, string_B, similarity]index_pairs:
string_pairs, but returns positions in your input list instead of the strings.[index_A, index_B, similarity]deduped_strings:
deduped_indices:
deduped_strings, but returns the indices of the kept items.membership_map:
[0,0,0,3,3] means rows 0/1/2 are one group (rep=0) and rows 3/4 are another (rep=3).row_annotations:
Returns one object per input row with an explanation of what it belongs to (rep row + similarity).
Use when: you want a human-readable, per-row result for debugging or UI display.
top_k (optional, integer or "all", default "all")
all = find all matches above threshold.
Or an integer (e.g., 50) to limit matches per row (faster, fewer results).
Sample request in python
import requests, json
API_KEY = "YOUR_ZYLA_KEY"
URL = "API_URL/dedupe"
data_list = ["Microsoft","Micsrosoft","Apple Inc","Apple","Google LLC","9oogle"]
params = {
"data": json.dumps(data_list),
"similarity_threshold": "0.75",
"remove_punctuation": "true",
"to_lowercase": "true",
"use_token_sort": "false",
"output_format": "string_pairs",
"top_k": "all"
}
headers = {"Authorization": f"Bearer {API_KEY}"}
r = requests.post(URL, headers=headers, params=params, timeout=60)
print(r.status_code)
print(r.json())
Dedupe - 端点功能
| 对象 | 描述 |
|---|---|
data |
[必需] JSON array of strings to deduplicate (max 1000). Example: ["a","b","c"] |
similarity_threshold |
可选 Similarity cutoff from 0 to 1. Higher values are stricter (fewer matches). Default is 0.75. |
remove_punctuation |
可选 If true, punctuation is removed before matching. Default is true. |
to_lowercase |
可选 If true, strings are lowercased before matching. Default is true. |
use_token_sort |
可选 If true, tokens in each string are sorted before matching. Useful when word order varies. Default is false. |
output_format |
可选 Default: string_pairs Allowed values (and what each means): index_pairs List of matches as [i, j, score] where i and j are indices in the input list. string_pairs List of matches as [string_i, string_j, score] using original strings. deduped_strings List of strings with duplicates removed (one representative per group). deduped_indices List of indices representing the deduplicated set (one representative per group). membership_map Array of length N where entry i is the representative index for the group of data[i]. row_annotations Array of objects (one per input row) with fields: index, original_string, rep_index, rep_string, similarity_to_rep. |
top_k |
可选 Limits how many neighbors are returned per input string. Use all for full dedupe, or a positive integer for top matches per row. |
{"status":"success","response_data":[["Apple","appl!e",1.0]]}
curl --location --request POST 'https://zylalabs.com/api/11916/preprocessed+record+similarity+api/22653/dedupe?data=["Apple", "appl!e"]' --header 'Authorization: Bearer YOUR_API_KEY'
| 标头 | 描述 |
|---|---|
授权
|
[必需] 应为 Bearer access_key. 订阅后,请查看上方的"您的 API 访问密钥"。 |
无长期承诺。随时升级、降级或取消。
去重端点返回一个 JSON 对象,包含匹配的字符串对、相似度分数和可选的去重结果。输出可以根据指定的配置格式化为字符串对、索引对或去重字符串
响应数据中的关键字段包括“状态”(指示成功或错误)和“响应数据”,其中包含根据用户请求格式化的结果,如匹配对或去重字符串
用户可以通过调整“config”对象中的参数来自定义请求,例如“similarity_threshold”用于匹配的严格性,“remove_punctuation”用于预处理,以及“output_format”以选择所需的结果结构
响应数据组织为一个结果数组,每个条目对应于一个匹配或去重字符串 根据输出格式条目可能包括原始字符串 索引和相似性分数 方便轻松集成到工作流程中
典型的用例包括去重客户名单 将记录与主名单对账 清理CRM数据 以及在不同数据源之间进行实体解析以确保数据的完整性和准确性
数据准确性通过先进的模糊匹配算法得以保持,这些算法考虑了常见的数据问题,如拼写错误和大小写差异。该API旨在有效处理混乱数据,确保可靠的匹配结果
接受的参数值包括“similarity_threshold”(0到1)、“remove_punctuation”(布尔值)、“to_lowercase”(布尔值)、“use_token_sort”(布尔值)和“top_k”(整数或“all”)。这些参数允许用户根据其特定需求调整匹配过程
如果去重端点返回部分或空结果,用户应检查输入数据是否存在质量问题,例如过多的重复项或非常低的相似性阈值。调整“相似性阈值”或审查输入列表可以帮助改善结果
服务级别:
100%
响应时间:
3,110ms
服务级别:
100%
响应时间:
1,937ms
服务级别:
100%
响应时间:
3,110ms
服务级别:
100%
响应时间:
3,110ms
服务级别:
100%
响应时间:
3,110ms
服务级别:
100%
响应时间:
1,695ms
服务级别:
100%
响应时间:
717ms
服务级别:
100%
响应时间:
704ms
服务级别:
100%
响应时间:
3,110ms
服务级别:
100%
响应时间:
1,072ms
服务级别:
100%
响应时间:
1,078ms
服务级别:
100%
响应时间:
814ms
服务级别:
100%
响应时间:
782ms
服务级别:
100%
响应时间:
1,029ms
服务级别:
100%
响应时间:
274ms
服务级别:
100%
响应时间:
239ms
服务级别:
100%
响应时间:
1,882ms
服务级别:
100%
响应时间:
266ms
服务级别:
100%
响应时间:
278ms
服务级别:
100%
响应时间:
863ms