PS-0 PropertyScraper
Autonomous multi-portal data engineering pipeline harvesting, normalizing, geocoding, and deduplicating Nigerian real estate listing telemetry in real-time.
Pipeline Execution Logs
Latest Run Telemetry by Mode
DISCOVERY RUN
SUCCESS
Total Ingested
14,218
New Discovered
+88
Updated
0
Duration
9m 12s
HEALTH CHECK RUN
SUCCESS
Checked
1,000
Active Verified
961
Removed
39
Price Changes
16
Real-Time Ingestion
Pipeline Volumes & Coverage
Total Records Ingested
14,218
Multi-portal cumulative scraped raw listings
Active Unique Listings
12,330
Verified active & deduplicated property listings
Spatial Coverage
10 Cities
468 distinct geocoded neighbourhoods
Source System Telemetry
Portal Ingestion Health Matrix
| Portal Target | Harvest Status | Listing Count | HTTP Success % | Avg Latency | Extractor Type |
|---|---|---|---|---|---|
| NigeriaPropertyCentre.ng | OPERATIONAL | 6,484 | 99.4% | 390 ms | HTML / BeautifulSoup |
| PropertyPro.ng | OPERATIONAL | 3,949 | 99.1% | 420 ms | HTML / BeautifulSoup |
| PrivateProperty.ng | OPERATIONAL | 3,785 | 98.8% | 580 ms | HTML / BeautifulSoup |
Pipeline Resilience
Data Hygiene & Quality Guardrails
Cross-Portal Deduplication
Multi-tiered hashing on normalized title, neighbourhood, price cohort, and spatial coordinates. Merges cross-posted listings with 14.2% duplicate merge rate across targets.
Outlier & Anomaly Detection
Automatic Z-score price filtering isolates extreme input errors (e.g. typos, zero-price listings). Filtered 42 price anomalies this month.
Adaptive Health Check Cooldown
Listing health check frequencies scale dynamically with listing age (2-day, 7-day, and 14-day intervals) driven by partial B-tree index queues and Overdue Ratio prioritization.