Text & Digit Normalization
- Introduction
- How Normalization Works
- Automatic Request Normalization
- Programmatic Usage
- PostgreSQL Trigram & Fuzzy Search Integration
- Testing Normalization
Introduction
In Iranian and regional e-commerce, inconsistencies in user keyboard layouts (such as mixed Arabic/Persian keyboards, Persian/Arabic numerals, and irregular zero-width non-joiners) frequently break phone number validations, National ID algorithms, and search query lookups.
Reyhan Commerce includes a dedicated, zero-dependency Text Normalization Engine that automatically cleans and standardizes incoming requests before they reach validation rules, Form Requests, or database queries.
How Normalization Works
Automatic Request Normalization
Every HTTP request to the /api/v1/* routes automatically executes through the normalization middleware stack:
- Digit Conversion: Translates all Persian (
۰-۹) and Arabic (٠-٩) numerals into standard ASCII digits (0-9). - Character Standardization: Unifies Arabic
يandكinto Persianیandک, and maps Arabicةtoه. - ZWNJ Sanitization: Strips redundant Zero-Width Non-Joiners and collapses irregular unicode spaces.
Programmatic Usage
You can invoke the normalization utilities anywhere in your application:
use Reyhan\Core\Services\Normalization\PersianNormalizer;
// 1. Convert numerals
$mobile = PersianNormalizer::toAsciiDigits('۰۹۱۲۳۴۵۶۷۸۹');
// Result: '09123456789'
// 2. Clean Persian typography
$title = PersianNormalizer::clean('كرم مرطوبكننده پوست');
// Result: 'کرم مرطوبکننده پوست'PostgreSQL Trigram & Fuzzy Search Integration
Normalized search strings are matched against PostgreSQL 17 using the pg_trgm extension and GIN indexing:
SELECT id, title, similarity(title, 'ضد آفتاب') AS score
FROM products
WHERE title % 'ضد آفتاب'
ORDER BY score DESC
LIMIT 20;Testing Normalization
<?php
declare(strict_types=1);
use Reyhan\Core\Services\Normalization\PersianNormalizer;
it('converts mixed arabic and persian numerals to ascii digits', function () {
expect(PersianNormalizer::toAsciiDigits('۰۹۱۲۳۴۵۶۷۸۹'))->toBe('09123456789')
->and(PersianNormalizer::toAsciiDigits('٠١٢٣٤٥٦٧٨٩'))->toBe('0123456789');
});
it('unifies arabic letters to standard persian', function () {
expect(PersianNormalizer::clean('كتاب كودك'))->toBe('کتاب کودک');
});