# Visitor Analytics

> 📘 Untuk panduan terperinci (Bahasa Melayu) — seni bina, aliran kod, laporan bulanan dan **cara paparkan statistik di frontend site** — lihat [`VISITOR-COUNTER-GUIDE.md`](./VISITOR-COUNTER-GUIDE.md).

## Overview

The Visitor module records public CMS page views for traffic analysis. It reports page views and pseudonymous unique visitors for today, this week, this month, and this year, and ranks the most-viewed pages for a selected period.

Analytics are stored in `visitor_page_views` and `visitor_daily_counters`. The legacy `visits` table, `Visit` model and visitor traits are no longer part of the codebase.

## What is recorded

The `visitor.log` middleware checks the two analytics switches independently and records data only when all of these request conditions are met:

- The request is `GET`.
- The response is HTTP `200` and has an HTML content type.
- The route is one of the public CMS routes (`home`, `frontend-site.home`, or `frontend-site.page`).

Each row contains a site key, a normalized page path, a pseudonymous visitor hash, and the timestamp. It does not store an IP address, raw User-Agent, request body, headers, or cookie token. The same visitor and page are recorded at most once in a rolling 30-minute window; visiting a different page still counts immediately. This reduces repeated reload inflation, but is not bot detection or a security control.

For the CMS `article-web` page, a valid published article's public `id` code is added to the analytics path (for example `/web/article-web/visi-misi`). Other query-string values are ignored, so article traffic is separated without recording arbitrary query parameters. Single-segment site home routes use their actual site key when they resolve to a site homepage.

The first-party `cms_visitor_id` cookie contains a random identifier, is HttpOnly and SameSite=Lax, and expires after `VISITOR_COOKIE_DAYS` (365 days by default). The database stores only an HMAC-SHA256 hash of that value, keyed with the Laravel application key. Rotating `APP_KEY` will cause existing browser cookies to be counted as new visitors. The cookie is refreshed on qualifying page views.

The Visitor admin screen has two independent switches:

- **Daily visitor counter** stores one pseudonymous visitor count per site per day in `visitor_daily_counters`. It continues to work if page-view tracking is paused.
- **Page-view analytics** stores the deduplicated page rows used by the traffic trend and Top Pages. It can be paused without turning off the daily counter.

Before enabling this in production, document the cookie and analytics in the site's privacy notice and confirm the retention and notice requirements with the organization's privacy/legal team. This implementation does not itself claim compliance with a particular legal or government standard.

## Metrics

- **Daily visits**: visitor-and-site days recorded in the period. A repeat visit to another page on the same site the same day does not add another daily count.
- **Unique visitors**: distinct pseudonymous hashes from daily counter rows when that counter is enabled; otherwise, distinct hashes from page-view rows if page-view tracking is enabled. This is not an IP-based or person-level count; clearing cookies, changing devices, or rotating the application key can result in a visitor being counted again.
- **Page Views**: number of qualifying visitor/page windows recorded in the period (repeated reloads of the same page by the same cookie within 30 minutes count once).
- **Top Pages**: page paths ranked by recorded page views, grouped by site key, for the selected report range (up to 25 rows).

Collection begins when the new migration is applied and tracking is enabled. The legacy `visits` table is no longer present.

## Data retention and cleanup

Retention can be configured in `config/visitor.php`:

```dotenv
VISITOR_COOKIE_DAYS=365
VISITOR_RETENTION_DAYS=365
```

The scheduled `visitor:prune` command deletes page-view and daily-counter rows older than the configured retention period. Run Laravel's scheduler in production (for example, through the host's normal scheduler/cron setup). On a local Windows/Laragon environment, run `php artisan schedule:work` when scheduled cleanup is needed.

The admin **Clear data** action deletes records from `visitor_page_views` and `visitor_daily_counters`.

## Legacy `visits` table (not part of the schema)

The legacy `visits` table, `Visit` model and the `HasVisits`/`Visitable` traits have been removed from the codebase. The visitor migration only defines the current schema:

- `visitor_page_views`
- `visitor_daily_counters`
- `visitor_settings.is_page_view_active`

If an older database still has a `visits` table, it is unused by the application. Back up the database, then drop it manually if desired:

```php
Schema::dropIfExists('visits');
```

Footer or dashboard content that still calls `Visit::` must be updated to `VisitorPageView` or the PortalHandler variables (see `VISITOR-COUNTER-GUIDE.md` §10).

## Admin page

The Visitor admin page is available at `/{ADMIN_PATH}/visitor` (default `portal-k7m4x9/visitor`, subject to the configured admin prefix and permissions). It provides:

- Independent daily-counter and page-view tracking switches.
- Four period cards (Today, This week, This month, This year) in one row on desktop, each showing its date range; separate metric cards summarise daily visits, unique visitors, page views and distinct pages.
- A traffic trend chart and a Top 10 page views chart.
- A confirmed action to clear visitor analytics records.
- Date-range, site and page-path filters. Page search filters page-view results; daily-counter figures remain site/date-based.
- A monthly report (calendar month + site) with KPI cards, Jan–Dec table, top pages and CSV/PDF export. The legacy top-pages CSV route (`visitor.export`) remains available for direct links, but its dashboard button has been removed.

## Public statistics page

A public-facing monthly statistics page mirrors the ministry portal's *Statistik Perkhidmatan Dalam Talian* page.

| Item | Value |
|------|-------|
| Page | `frontend_pages` #40 · slug `visitor-statistics` · URL `/nres/visitor-statistics` |
| Content | `storage/app/private/PAGE_CONTENT/nres/visitor-statistics.blade.php` (rendered from `page_content`) |
| Scope | **Global** — all portals (`$vsScopeMode = 'global'` in the page content; set `'site'` to scope to the current portal only) |
| Data | `visitor_daily_counters` (visits / unique) · `visitor_page_views` (page views, by `page_path`) |
| Friendly names | `frontend_pages.name` matched by `page_path` → slug |
| Layout | Self-contained — own `<style>` (`.vs-*` classes) + markup; BM/EN by session locale |

> ℹ️ **Scope (updated 2026-10-25).** The page now runs in **global** mode (`$vsScopeMode = 'global'`), so it
> aggregates every portal — matching the footer stats (`$totalVisits`, `$thisYearVisits`,
> `$uniqueVisitors` from `PortalHandler`, which are always global). Set `$vsScopeMode = 'site'` inside the
> page content to scope it back to `site_key = site:{id}` of the current portal (resolved from the URL slug,
> never hard-coded).
>
> Because both sides are global, a deleted site's leftover rows can no longer make the numbers drift — but
> they still inflate the totals, so clean them up if a site is removed outside the CMS:
>
> ```bash
> php storage/app/cms-page-content/cleanup-orphan-site-keys.php --dry-run   # senarai sahaja
> php storage/app/cms-page-content/cleanup-orphan-site-keys.php             # backup + padam
> ```
>
> `FrontendSiteController::destroy()` already removes the visitor rows for the site being deleted.
> Verified 2026-10-25: inserting a `site:999` row made the page show it (+1 view, +1 unique, +1 service),
> then removing it restored the counts.


**Tables**

1. *Bilangan Pelawat Portal Rasmi Kementerian* — rows: Bilangan Pelawat · Pelawat Unik · Paparan Halaman; columns Jan…Dis + Jumlah.
2. *Perkhidmatan Dalam Talian (Halaman Paling Dilawati)* — one row per `page_path` (friendly page name + path subtitle), sorted by total views.

**Metric cards**

| Card | Source |
|------|--------|
| Jumlah Pelawat | `COUNT(*)` on `visitor_daily_counters` for the year |
| Pelawat Unik | `COUNT(DISTINCT visitor_hash)` for the year |
| Paparan Halaman | `COUNT(*)` on `visitor_page_views` for the year |
| Perkhidmatan Dalam Talian | distinct `page_path` count for the year |

The year is auto-detected from the latest `viewed_at` (falls back to the current year), so the page rolls over on its own.

**Scripts:** `storage/app/cms-page-content/template-visitor-statistics.txt` → `apply-visitor-stats.php` (writes `page_content`, runs `BladeSyncService::syncPageContent()`; `--dry-run`; backup `backups/page40-visitor-statistics-backup-*.json`).

> ⚠️ **`apply-*.php` writes the `.txt` template VERBATIM** — so a stale template silently reverts the page
> (losing `t()` keys, scope changes, etc.). Keep templates in sync with the live page:
>
> ```bash
> php storage/app/cms-page-content/sync-templates-from-live.php --dry-run   # tunjuk beza
> php storage/app/cms-page-content/sync-templates-from-live.php             # selaraskan + backup
> ```
>
> It covers `article` · `faq` · `location` · `officer-directory` · `visitor-statistics` · `gallery` ·
> `publication` · `speech` · `calendar`→`calender` · `archive`→`electronic-archive` · `media`→`media-statement` ·
> `tender`→`tenders-and-quotations`. Verified 2026-10-25: **12/12** templates match their live pages and
> `apply-visitor-stats.php --dry-run` reports `10461 → 10461 bait` (no change).


## Main files

| File | Responsibility |
|------|----------------|
| `app/Http/Middleware/VisitorLogMiddleware.php` | Eligibility checks and cookie handling |
| `app/Services/VisitorPageViewTracker.php` | Site/path normalization and pseudonymous hash |
| `app/Models/Backend/VisitorPageView.php` | Analytics queries |
| `app/Models/Backend/VisitorDailyCounter.php` | Daily visitor counter records |
| `app/Http/Controllers/Backend/VisitorController.php` | Admin summary, top pages, toggle and clear |
| `resources/views/backend/module/visitor/index.blade.php` | Visitor analytics UI |
| `database/migrations/2026_10_20_000000_consolidate_visitor_analytics_schema.php` | Consolidated visitor schema (page views, daily counters, tracking flag) |
| `config/visitor.php` | Cookie and retention settings |
| `storage/app/private/PAGE_CONTENT/nres/visitor-statistics.blade.php` | Public monthly statistics page (page #40) |
| `storage/app/cms-page-content/apply-visitor-stats.php` | Applies `template-visitor-statistics.txt` to page #40 |
| `routes/console.php` | Scheduled retention cleanup |
