Commit fe003fd4 authored by Mahmoud Aglan's avatar Mahmoud Aglan

fix(settlements): find the name, count the money, and stop crying wolf

Three things the screen got wrong on the first day it was used.

Search could not find people. `like '%term%'` over name_ar fails on this
data for two reasons that have nothing to do with the searcher being
careless: nobody agrees about hamza (عبدالله أحمد / عبدالله احمد is the
same child, so is يحيى/يحيي and حمزة/حمزه), and a name on file is four or
five words while the person searching types the two they remember —
"عبدالله صلاح" against "عبدالله أحمد صلاح سيد" matches nothing because
those words are not adjacent. ArabicSearch folds both sides to one
spelling and requires each word of the term to appear somewhere in the
name, folding in SQL (Postgres translate) so the database does the work.

The bundle probe asked whether a product line existed, so a boy who paid
2,500 toward his federation card — typed as "القسط الاول" on the same
invoice as his kit — was reported as never having bought one. That is the
exact reading the group roster stopped doing last week, and two screens
answering the same question differently is worse than either answer. It
now reads the money the way BundledProductLine does, bare instalments
included where the programme requires exactly one product, and reports a
position rather than a yes/no: paid, part paid with the remainder and a
progress bar, or nothing at all.

And the worklist was flagging ordinary business. A renewal issued on the
1st and due on the 8th is not an anomaly, it is Tuesday — so the flags now
fire on invoices past their due date, not merely unpaid. A card being paid
off on an agreed plan through the till is not an anomaly either; only
money recorded outside the product is. "Requires a card and has not bought
one" is a sales fact, not a payment anomaly, so it annotates an account
without summoning it. And enrolment start_date is copied from the GROUP's
season start, so a player entered in August carried a 16 July start and
was reported as owing months of a season he was not in — the month he
joined is the later of start_date and enrollment_date.

On the restored tenant this takes the worklist from 165 accounts to 77,
and unbilled-month flags from 35 to 1. Participant 219 now reads
"سدد 2,500 من 8,000" instead of "لم يُحاسَب على مستلزم البرنامج".

Verified: 46 settlement/search/render cases pass, full suite 318 tests on
both SQLite and the restored tenant, no failures.
Co-Authored-By: 's avatarClaude Opus 5 (1M context) <noreply@anthropic.com>
parent 20c5cf15
<?php
namespace App\Domain\Shared\Support;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Support\Facades\DB;
/**
* Searching for a person by their Arabic name, the way the name was actually
* typed.
*
* `where('name_ar', 'like', "%{$term}%")` fails on this data constantly, for two
* reasons that have nothing to do with the searcher being careless:
*
* Spelling. "عبدالله أحمد" and "عبدالله احمد" are the same child; so are
* "يحيى"/"يحيي" and "حمزة"/"حمزه". Whoever typed the row and whoever is
* searching for it rarely agree about hamza, ة/ه or ى/ي, and neither of them
* is wrong.
*
* Word order and gaps. A name on file is four or five words —
* "عبدالله أحمد صلاح سيد" — and the person searching types the two they
* remember, "عبدالله صلاح". A single LIKE over the whole phrase matches
* nothing, because those two words are not adjacent.
*
* So both sides are folded to one spelling and the term is split into words,
* each of which must appear somewhere in the name. The folding happens in SQL
* too (Postgres `translate`, character for character) so the database does the
* matching rather than the application pulling every participant into memory.
*/
final class ArabicSearch
{
/**
* Letter variants that mean the same name. Both strings must have the same
* number of characters: translate() maps them one to one.
*/
private const FROM = 'أإآٱىئةؤ';
private const TO = 'ااااييهو';
/** Fold a search term or a stored name to the one spelling both are compared in. */
public static function normalise(?string $text): string
{
if ($text === null || $text === '') {
return '';
}
// Tashkeel and tatweel carry no meaning for matching.
$text = preg_replace('/[\x{064B}-\x{0652}\x{0640}\x{0670}]/u', '', $text) ?? '';
$map = [];
$from = preg_split('//u', self::FROM, -1, PREG_SPLIT_NO_EMPTY);
$to = preg_split('//u', self::TO, -1, PREG_SPLIT_NO_EMPTY);
foreach ($from as $i => $char) {
$map[$char] = $to[$i];
}
// Arabic-Indic digits, so a phone typed as ٠١٠٠ finds 0100.
$map += [
'٠' => '0', '١' => '1', '٢' => '2', '٣' => '3', '٤' => '4',
'٥' => '5', '٦' => '6', '٧' => '7', '٨' => '8', '٩' => '9',
'۰' => '0', '۱' => '1', '۲' => '2', '۳' => '3', '۴' => '4',
'۵' => '5', '۶' => '6', '۷' => '7', '۸' => '8', '۹' => '9',
];
$text = strtr($text, $map);
$text = preg_replace('/\s+/u', ' ', $text) ?? '';
return mb_strtolower(trim($text));
}
/** The words a term is made of, each of which must appear in the name. */
public static function words(?string $term): array
{
$normalised = self::normalise($term);
if ($normalised === '') {
return [];
}
return array_values(array_filter(explode(' ', $normalised), fn ($w) => $w !== ''));
}
/**
* Apply the search to a query over `people`, folding the stored names the
* same way.
*
* Only Postgres has translate(); on any other driver (the SQLite test
* suite) the folding is skipped and a plain word-wise LIKE is used, which
* still fixes the word-order half of the problem.
*
* @param array<int, string> $columns qualified name columns, e.g. people.name_ar
*/
public static function apply(Builder $query, string $term, array $columns, ?string $phoneColumn = null): Builder
{
$words = self::words($term);
if ($words === []) {
return $query;
}
$folds = DB::connection()->getDriverName() === 'pgsql';
foreach ($words as $word) {
$query->where(function (Builder $inner) use ($word, $columns, $phoneColumn, $folds) {
foreach ($columns as $column) {
$expression = $folds
? DB::raw("lower(translate({$column}, '" . self::FROM . "', '" . self::TO . "'))")
: DB::raw("lower({$column})");
$inner->orWhere($expression, 'like', '%' . $word . '%');
}
if ($phoneColumn) {
$inner->orWhere($phoneColumn, 'like', '%' . $word . '%');
}
});
}
return $query;
}
/** Does this stored name match the term? Same rule, for in-memory lists. */
public static function matches(?string $haystack, string $term): bool
{
$name = self::normalise($haystack);
foreach (self::words($term) as $word) {
if (! str_contains($name, $word)) {
return false;
}
}
return true;
}
}
...@@ -9,6 +9,7 @@ ...@@ -9,6 +9,7 @@
use App\Domain\Participant\Models\Participant; use App\Domain\Participant\Models\Participant;
use App\Domain\Pricing\Services\PricingService; use App\Domain\Pricing\Services\PricingService;
use App\Domain\Shared\Exceptions\DomainException; use App\Domain\Shared\Exceptions\DomainException;
use App\Domain\Shared\Support\ArabicSearch;
use Illuminate\Support\Carbon; use Illuminate\Support\Carbon;
use Livewire\Attributes\Layout; use Livewire\Attributes\Layout;
use Livewire\Attributes\Locked; use Livewire\Attributes\Locked;
...@@ -122,12 +123,16 @@ public function searchResults() ...@@ -122,12 +123,16 @@ public function searchResults()
return collect(); return collect();
} }
$term = '%' . trim($this->search) . '%'; // Folded spelling and word-wise matching: "عبدالله صلاح" has to find
// "عبدالله أحمد صلاح سيد", and "احمد" has to find "أحمد". See
// ArabicSearch for why a plain LIKE cannot.
return Participant::with(['person', 'enrollments.program']) return Participant::with(['person', 'enrollments.program'])
->whereHas('person', fn ($q) => $q->where('name_ar', 'like', $term) ->whereHas('person', fn ($q) => ArabicSearch::apply(
->orWhere('name', 'like', $term) $q,
->orWhere('phone', 'like', $term)) $this->search,
['people.name_ar', 'people.name'],
'people.phone',
))
->limit(15) ->limit(15)
->get(); ->get();
} }
......
...@@ -3,6 +3,7 @@ ...@@ -3,6 +3,7 @@
namespace App\Livewire\Admin; namespace App\Livewire\Admin;
use App\Domain\Financial\Services\AccountAnomalyScanner; use App\Domain\Financial\Services\AccountAnomalyScanner;
use App\Domain\Shared\Support\ArabicSearch;
use Livewire\Attributes\Layout; use Livewire\Attributes\Layout;
use Livewire\Attributes\Title; use Livewire\Attributes\Title;
use Livewire\Attributes\Url; use Livewire\Attributes\Url;
...@@ -63,9 +64,11 @@ private function rows(): array ...@@ -63,9 +64,11 @@ private function rows(): array
return array_values(array_filter($rows, function ($row) use ($term) { return array_values(array_filter($rows, function ($row) use ($term) {
$person = $row['participant']->person; $person = $row['participant']->person;
return str_contains((string) $person?->name_ar, $term) // Same folding the wizard's search uses, so a name found there is
|| str_contains((string) $person?->name, $term) // found here too.
|| str_contains((string) $person?->phone, $term); return ArabicSearch::matches($person?->name_ar, $term)
|| ArabicSearch::matches($person?->name, $term)
|| str_contains((string) $person?->phone, ArabicSearch::normalise($term));
})); }));
} }
......
...@@ -101,7 +101,14 @@ class="text-sm text-amber-700 hover:text-amber-900 font-medium"> ...@@ -101,7 +101,14 @@ class="text-sm text-amber-700 hover:text-amber-900 font-medium">
@endforeach @endforeach
<div class="p-3 bg-gray-50 rounded-lg"> <div class="p-3 bg-gray-50 rounded-lg">
<p class="text-[11px] text-gray-500">{{ __('فواتير غير مسددة') }}</p> <p class="text-[11px] text-gray-500">{{ __('فواتير غير مسددة') }}</p>
<p class="text-base font-bold text-gray-900 tabular-nums">{{ $money['unpaid'] ?? 0 }}</p> <p class="text-base font-bold text-gray-900 tabular-nums">
{{ $money['unpaid'] ?? 0 }}
@if(($money['overdue'] ?? 0) > 0)
<span class="text-xs font-normal text-red-700">({{ $money['overdue'] }} {{ __('متأخرة') }})</span>
@endif
</p>
{{-- A renewal issued this month and not yet due is ordinary
business; only what is past its date is counted late. --}}
</div> </div>
</div> </div>
</div> </div>
...@@ -109,11 +116,12 @@ class="text-sm text-amber-700 hover:text-amber-900 font-medium"> ...@@ -109,11 +116,12 @@ class="text-sm text-amber-700 hover:text-amber-900 font-medium">
{{-- What the system thinks is wrong --}} {{-- What the system thinks is wrong --}}
@php @php
$flags = []; $flags = [];
if (($money['paid'] ?? 0) === 0 && ($money['unpaid'] ?? 0) > 0) $flags[] = 'never_paid'; if (($money['paid'] ?? 0) === 0 && ($money['overdue'] ?? 0) > 0) $flags[] = 'never_paid';
if (($money['unpaid'] ?? 0) >= 2) $flags[] = 'stacked_unpaid'; if (($money['overdue'] ?? 0) >= 2) $flags[] = 'stacked_unpaid';
if (($money['zero_invoices'] ?? 0) > 0) $flags[] = 'zero_invoice'; if (($money['zero_invoices'] ?? 0) > 0) $flags[] = 'zero_invoice';
if (!empty($diagnosis['handtyped'])) $flags[] = 'handtyped_product'; if (!empty($diagnosis['handtyped'])) $flags[] = 'handtyped_product';
if (!empty($diagnosis['missing_bundle'])) $flags[] = 'missing_bundle'; if (!empty($diagnosis['missing_bundle'])) $flags[] = 'missing_bundle';
if (!empty($diagnosis['partial_bundle'])) $flags[] = 'partial_bundle';
if (!empty($diagnosis['unbilled_months'])) $flags[] = 'unbilled_month'; if (!empty($diagnosis['unbilled_months'])) $flags[] = 'unbilled_month';
if (!empty($diagnosis['duplicate_of'])) $flags[] = 'duplicate_person'; if (!empty($diagnosis['duplicate_of'])) $flags[] = 'duplicate_person';
@endphp @endphp
...@@ -240,19 +248,56 @@ class="text-sm text-amber-700 hover:text-amber-900 font-medium"> ...@@ -240,19 +248,56 @@ class="text-sm text-amber-700 hover:text-amber-900 font-medium">
</div> </div>
@endif @endif
{{-- Bundle never charged --}} {{-- Where he stands on what the programme requires --}}
@if(!empty($diagnosis['missing_bundle'])) @if(!empty($diagnosis['bundle_status']))
<div class="bg-white border border-gray-200 rounded-xl p-4 mb-4"> <div class="bg-white border border-gray-200 rounded-xl p-4 mb-4">
<h3 class="text-sm font-bold text-gray-900 mb-1">{{ __('مستلزمات البرنامج غير المحاسَب عليها') }}</h3> <h3 class="text-sm font-bold text-gray-900 mb-1">{{ __('مستلزمات البرنامج') }}</h3>
<p class="text-xs text-gray-500 mb-3">{{ __('البرنامج يتطلب هذه المنتجات ولا يوجد لها أي مبلغ على حساب المشترك.') }}</p> <p class="text-xs text-gray-500 mb-3">
<div class="flex flex-wrap gap-2"> {{ __('يُحتسب المدفوع من الفواتير المجمّعة أيضاً لو دفع القسط ضمن فاتورة فيها الزي، حصته من الدفع محسوبة هنا.') }}
@foreach($diagnosis['missing_bundle'] as $bundle) </p>
<button type="button" class="px-2.5 py-1.5 text-xs rounded-lg bg-gray-50 border border-gray-200 hover:bg-amber-50 hover:border-amber-200" <ul class="space-y-2">
wire:click="startDraft('sell_product', {{ \Illuminate\Support\Js::from($bundle) }})"> @foreach($diagnosis['bundle_status'] as $bundle)
{{ $bundle['product_name'] }} <span class="text-gray-400" dir="ltr">· {{ format_money($bundle['price']) }}</span> @php
$percent = $bundle['expected'] > 0
? min(100, (int) round($bundle['paid'] * 100 / $bundle['expected']))
: 0;
$remaining = max(0, $bundle['expected'] - $bundle['paid']);
@endphp
<li class="p-3 rounded-lg border {{ $bundle['status'] === 'paid' ? 'border-emerald-200 bg-emerald-50/50' : ($bundle['status'] === 'partial' ? 'border-amber-200 bg-amber-50/50' : 'border-red-200 bg-red-50/40') }}">
<div class="flex flex-wrap items-center justify-between gap-2">
<div class="min-w-0">
<p class="text-sm font-medium text-gray-900">{{ $bundle['product_name'] }}</p>
<p class="text-xs mt-0.5 {{ $bundle['status'] === 'paid' ? 'text-emerald-700' : ($bundle['status'] === 'partial' ? 'text-amber-700' : 'text-red-700') }}">
@if($bundle['status'] === 'paid')
{{ __('مدفوع بالكامل') }}
@elseif($bundle['status'] === 'partial')
{{ __('سدد') }} <span dir="ltr">{{ format_money($bundle['paid']) }}</span>
{{ __('من') }} <span dir="ltr">{{ format_money($bundle['expected']) }}</span>
({{ $percent }}%) — {{ __('متبقٍ') }} <span dir="ltr">{{ format_money($remaining) }}</span>
@else
{{ __('لا يوجد له أي مبلغ') }} · <span dir="ltr">{{ format_money($bundle['price']) }}</span>
@endif
@if($bundle['from_text'])
<span class="text-gray-500">· {{ __('مسجَّل كبند يدوي') }}@if($bundle['inferred']) ({{ __('قسط بلا اسم') }})@endif</span>
@endif
</p>
</div>
@if($bundle['status'] !== 'paid')
<button type="button" class="px-2.5 py-1.5 text-xs rounded-lg bg-white border border-gray-300 hover:bg-amber-50 hover:border-amber-300 shrink-0"
wire:click="startDraft('sell_product', {{ \Illuminate\Support\Js::from(['product_id' => $bundle['product_id'], 'price' => $remaining ?: $bundle['price']]) }})">
{{ $bundle['status'] === 'partial' ? __('تسجيل باقي القيمة') : __('تسجيل البيع') }}
</button> </button>
@endforeach @endif
</div> </div>
@if($bundle['expected'] > 0 && $bundle['status'] !== 'missing')
<div class="mt-2 h-1.5 w-full bg-gray-200 rounded-full overflow-hidden">
<div class="h-full rounded-full {{ $bundle['status'] === 'paid' ? 'bg-emerald-500' : 'bg-amber-500' }}"
style="width: {{ max($percent, $bundle['paid'] > 0 ? 4 : 0) }}%"></div>
</div>
@endif
</li>
@endforeach
</ul>
</div> </div>
@endif @endif
......
...@@ -566,6 +566,63 @@ public function test_the_scanner_finds_accounts_and_explains_each_one(): void ...@@ -566,6 +566,63 @@ public function test_the_scanner_finds_accounts_and_explains_each_one(): void
$this->assertSame($sorted, $severities); $this->assertSame($sorted, $severities);
} }
public function test_this_months_renewal_is_not_an_anomaly(): void
{
// The screen exists for files a person has to decide about. A renewal
// issued on the 1st and due on the 8th is ordinary business that the
// collect-payment screen handles; flagging it buries everything else.
$participant = Participant::withoutGlobalScopes()
->whereHas('enrollments', fn ($q) => $q->where('status', 'active'))
->with(['enrollments' => fn ($q) => $q->where('status', 'active')])
->firstOrFail();
$before = app(AccountAnomalyScanner::class)->forParticipant($participant)['money'];
$invoice = Invoice::withoutGlobalScopes()->create([
'academy_id' => $participant->academy_id,
'branch_id' => $participant->branch_id,
'number' => 'INV-TEST-' . uniqid(),
'billable_type' => $participant->getMorphClass(),
'billable_id' => $participant->id,
'subtotal_amount' => 90000,
'total_amount' => 90000,
'paid_amount' => 0,
'due_amount' => 90000,
'issue_date' => now()->startOfMonth()->toDateString(),
'due_date' => now()->addWeek()->toDateString(),
'status' => InvoiceStatus::Sent,
'currency' => 'EGP',
]);
$after = app(AccountAnomalyScanner::class)->forParticipant($participant)['money'];
$this->assertSame($before['overdue'], $after['overdue'], 'A renewal that is not due yet is not late.');
$this->assertSame($before['unpaid'] + 1, $after['unpaid'], 'It is still shown as outstanding.');
$this->assertGreaterThan(0, $invoice->id);
}
public function test_a_card_paid_in_a_combined_invoice_reads_as_part_paid_not_missing(): void
{
// Production participant 219: 2,500 toward the federation card, typed
// as "القسط الاول" on the same invoice as his kit. The screen used to
// say he had never bought one.
$participant = Participant::withoutGlobalScopes()->find(219);
if (! $participant) {
$this->markTestSkipped('This tenant copy does not carry participant 219.');
}
$diagnosis = app(AccountAnomalyScanner::class)->forParticipant($participant);
$card = collect($diagnosis['bundle_status'])->firstWhere('product_id', 2);
$this->assertNotNull($card, 'His programme requires the federation card.');
$this->assertSame('partial', $card['status'], 'Money on a combined invoice still counts.');
$this->assertSame(250000, $card['paid']);
$this->assertSame(800000, $card['expected'], 'The total is the price for his membership tier.');
$this->assertTrue($card['from_text']);
$this->assertSame([], $diagnosis['missing_bundle'], 'He is not missing it — he is paying it off.');
}
public function test_the_diagnosis_answers_the_wizards_questions(): void public function test_the_diagnosis_answers_the_wizards_questions(): void
{ {
$participant = $this->participantWithDue(); $participant = $this->participantWithDue();
......
<?php
namespace Tests\Unit;
use App\Domain\Shared\Support\ArabicSearch;
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;
/**
* Finding a child by name, the way people actually type it.
*
* Every pair below is a real name from a production academy and a search that
* failed against it before this existed.
*/
class ArabicSearchTest extends TestCase
{
private const NAME = 'عبدالله أحمد صلاح سيد';
#[DataProvider('termsThatMustFind')]
public function test_a_name_is_found_however_it_is_typed(string $term): void
{
$this->assertTrue(ArabicSearch::matches(self::NAME, $term), "[{$term}] should find the name");
}
public static function termsThatMustFind(): array
{
return [
'exactly as stored' => ['عبدالله أحمد صلاح سيد'],
'without the hamza' => ['عبدالله احمد'],
'two words that are not adjacent' => ['عبدالله صلاح'],
'the last two words' => ['صلاح سيد'],
'one word' => ['صلاح'],
'extra spaces' => [' عبدالله سيد '],
'reversed order' => ['سيد عبدالله'],
];
}
public function test_a_different_child_is_not_found(): void
{
$this->assertFalse(ArabicSearch::matches(self::NAME, 'عبدالرحمن'));
$this->assertFalse(ArabicSearch::matches(self::NAME, 'عبدالله فريد'));
}
#[DataProvider('spellingPairs')]
public function test_spelling_variants_fold_together(string $a, string $b): void
{
$this->assertSame(ArabicSearch::normalise($a), ArabicSearch::normalise($b));
}
public static function spellingPairs(): array
{
return [
'hamza on alef' => ['أحمد', 'احمد'],
'hamza under alef' => ['إبراهيم', 'ابراهيم'],
'madda' => ['آدم', 'ادم'],
'ta marbuta' => ['حمزة', 'حمزه'],
'alef maqsura' => ['يحيى', 'يحيي'],
'hamza on ya' => ['سائد', 'سايد'],
'tashkeel' => ['مُحَمَّد', 'محمد'],
'arabic-indic digits' => ['٠١٠٠٣٠٥٥٥١', '0100305551'],
];
}
public function test_an_empty_term_matches_everything(): void
{
// An empty search box must not filter the list to nothing.
$this->assertSame([], ArabicSearch::words(''));
$this->assertTrue(ArabicSearch::matches(self::NAME, ''));
}
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment