LeadGen: Local Research Automation

LeadGen is a local research dashboard for collecting and reviewing public business information. It is built around a practical workflow: choose a niche and city, run browser-driven source collection, normalize records, review them locally, and optionally generate outreach angles with Ollama.

The point is not "scraping" as a buzzword. The point is a repeatable operator workflow with logs, source status, local records, and reviewable output.

Problem

Manual lead research is repetitive and hard to audit. A person can find businesses one by one, but the process gets messy fast: duplicate sources, inconsistent names, missing websites, dead pages, and no clean run history.

LeadGen turns that into a local scan and review loop.

What I Built

1. Scanner dashboard: Niche, city, suburb, proxy, and model controls for launching research runs.
2. Playwright automation: Browser-driven source collection for public directories and map/search sources.
3. Local review CRM: A table of normalized records with phone, source, city, score, status, website status, notes, and tags.
4. Analytics: Source performance, city performance, scan history, hit rate, and total lead summaries.
5. Local AI assist: Ollama-backed pitch angle generation so review help can stay on the machine.
6. Security architecture: Payment webhooks, admin operations, and account-state changes are handled as explicit trust-boundary decisions instead of default route plumbing.

Security Architecture

LeadGen has a small but real security architecture around payment events, admin actions, and account status.

• Webhook-to-database boundary: internalMutation is the trust boundary between HTTP webhooks and database writes. The public webhook receives and verifies the external event; the internal mutation owns the state change.
• Server-side admin checks: Admin operations perform role checks on the server for each action. They do not rely only on route-level guards or hidden UI controls.
• Payment webhook verification: The payment webhook uses timing-safe bearer token comparison so token validation does not leak useful comparison timing.
• Chargeback escalation: Chargebacks move through explicit account status states instead of a loose boolean flag, which makes account restrictions and review paths easier to audit.

Constraints

• Public sources behave differently and can rate-limit, time out, or return empty results.
• Browser automation needs cancellation, logging, and recovery when navigation fails.
• The operator still needs review control; raw automated output should not be treated as final truth.
• Local-first storage keeps the work inspectable and avoids sending every record to a hosted model.
• Webhook handlers should verify external input before invoking internal database mutations.
• Admin tools should keep server-side authorization checks close to the mutation or action being performed.

Proof Artifacts

The project includes screenshots of the scanner view, live feed, CRM table, analytics, duplicate review, and generated pitch angle. The visible run logs show source-by-source progress, failures, fallbacks, and normalized output instead of hiding the automation behind a vague success message.

Example proof shown in the interface:

• Home Services in Miami Florida scan.
• 70 total leads in the CRM.
• Yellow Pages source performance.
• Local status labels for "new" and "No website."
• Ollama-ready model control.

Tradeoffs

The system favors local review and traceability over invisible automation. That means the UI exposes messy source behavior: timeouts, empty responses, and navigation failures. For a research tool, that is useful because the operator can see what actually happened during the run.

Technical Direction

The next useful improvements are better deduplication, source configuration, richer run replay, and a clearer review workflow for turning discovered records into outreach-ready batches.