Dubai Flea Market Intelligence

Overview

This bot monitors dubai-fleamarket.com around the clock and sends instant Telegram notifications the moment a new event is posted — so enthusiasts never have to check the site manually again. It scrapes every 15 minutes, stores what it has seen, and only alerts on genuinely new listings, with an admin dashboard for analytics.

The Problem

Dubai Flea Market events are popular, capacity-limited, and announced on no fixed schedule — with no official notification system. By the time most people hear about a new event through word of mouth or a casual site visit, spots are already filling up. Manually refreshing the page every few hours is unreliable and exhausting, and missing an announcement means missing the event itself.

The Solution

A cloud bot polls the site every 15 minutes, diffs the response against a database of events it has already seen, and pushes an instant Telegram alert — event name, date, and a direct link — the moment something genuinely new appears. An email fallback queue backs up delivery, and a full admin dashboard provides live status, analytics, logs, and one-click controls.

Key Features

• Polls the source site every 15 minutes, 24/7
• Instant Telegram alerts on newly posted events only (no duplicates)
• Admin dashboard with overview and analytics
• Persists seen events so restarts never re-notify

Tech Stack & Why

• Python — The natural choice for scraping — mature parsing libraries and quick to iterate.
• Flask — A minimal web layer for the admin dashboard without dragging in a heavy framework.
• SQLite — Zero-config persistence — perfect for a single-purpose bot that just needs to remember what it has seen.
• Telegram Bot API — Free, instant push to where the audience already is, no app to install.
• Render — Easy always-on hosting for a long-running background worker.

Biggest Challenge

Reliable, free, always-on delivery. Render's free tier sleeps on inactivity and Telegram can rate-limit, so any announcement could be silently lost. The fix was a single-process daemon-thread poller, an UptimeRobot keep-alive ping to stay awake 24/7, and a Telegram→email fallback queue with automatic retry — with the database (not the source site) treated as the source of truth for idempotent "have I seen this before?" diffing.

Results & Impact

• Runs 24/7 — ~96 automated checks per day with zero manual effort
• Sub-minute Telegram alerts the moment an event is posted
• Permanent, searchable history of every event detected
• Built over ~4 months across 900+ commits — live in production

What I Learned

Writing a scraper that runs unattended forced me to think about reliability — handling site changes, avoiding duplicate alerts, and recovering from downtime. Idempotency (never notifying twice) turned out to be the feature that made it trustworthy.