Mastering `marketplace-report-crawler`: The Complete Guide

In the era of Vibe Coding, speed is the only metric that truly matters. You have an idea, you prompt your agent, and within minutes, you have a functional prototype. But there is a silent killer of great products: building something nobody actually wants.

The “Vibe” in Vibe Coding isn’t just about the aesthetics of your code or the smoothness of your UI; it’s about the resonance between your product and the market. Traditionally, market research was a grueling, manual process of scrolling through app stores, copy-pasting reviews into spreadsheets, and trying to spot patterns in a sea of noise. By the time you finished your analysis, the window of opportunity had closed.

Enter the marketplace-report-crawler. This isn’t just a scraping script; it is a high-fidelity sensory organ for your autonomous development environment. It allows you to transform raw marketplace data into actionable “Vibes” that fuel your AI’s creative process. This guide will walk you through mastering this skill to ensure your next build isn’t just fast, but inevitable.

The Core Problem: The Data-Intuition Gap

Vibe Coding relies on the LLM’s ability to “hallucinate” a solution that fits your intent. However, if your intent is based on thin air, the LLM will build a perfect solution for a non-existent problem. Most developers fail here because they lack the “ground truth” of what users are actually complaining about, paying for, or searching for in existing marketplaces (App Store, Google Play, Shopify, Chrome Web Store, etc.).

The marketplace-report-crawler solves this by automating the Job-To-Be-Done (JTBD) of market discovery. It “hires” autonomous agents to go where your users are, listen to their frustrations, and report back in a format that your coding agents (like Gemini or Claude) can consume instantly.

How It Works: The Architecture of Intelligence

The crawler operates on a three-tier execution model:

Targeting & Discovery: You define the “Marketplace Perimeter.” Whether it’s a specific category (e.g., “AI Productivity Tools”) or a competitor’s URL, the crawler maps the landscape.
Contextual Extraction: Unlike generic scrapers that just grab text, the marketplace-report-crawler uses semantic filtering to identify:
- Feature Requests: “I wish this app had X.”
- Pain Points: “It crashes whenever I try to do Y.”
- Emotional Highs: “I love how simple Z is.”
- Pricing Sentiment: “Too expensive for what it offers.”
Synthesis & Vibe Injection: The raw data is compressed into a MARKET_VIBE.md file. This file acts as a “System Prompt Supplement” for your coding tools, ensuring every line of code written is anchored in real-world demand.

Practical Example: Disrupting the “Minimalist Habit Tracker” Niche

Let’s say you want to build a new habit tracker. The market is saturated. If you just ask an AI to “build a habit tracker,” it will give you a clone of everything else. Instead, let’s use the crawler to find the “Vibe Gap.”

Step 1: Initializing the Crawl

First, we trigger the skill within our workspace. We aren’t just looking for one app; we’re looking for the top 10 competitors in the “Minimalist Productivity” category of the Chrome Web Store.

# Example command (Conceptual CLI usage)
cm skill marketplace-report-crawler --target "chrome-web-store" --query "habit tracker" --depth 10 --output ./docs/market-research/

Step 2: Analyzing the “Signal”

The crawler goes to work. It bypasses the shiny marketing copy and dives straight into the Reviews and Version History. It notices a recurring pattern across three different apps:

User A: “I love the UI, but I hate that I can’t sync this with my terminal.”
User B: “Too many notifications. I just want a silent log.”
User C: “The ‘minimalism’ is a lie; there are 50 settings I don’t need.”

Step 3: Generating the Vibe Report

The crawler synthesizes this into a report. Here is what the output looks like:

Vibe Gap Identified: The “Invisible” Tracker

Current Market Failure: Over-complication disguised as minimalism.

Primary Pain Point: Friction between “Work Context” (Terminal/IDE) and “Habit Context” (Browser/Mobile).

Actionable Instruction: Build a habit tracker that lives in the CLI but syncs to a beautiful, read-only web dashboard. Zero notifications. All interactions must be < 2 seconds.

Step 4: Injecting the Vibe into the Codebase

Now, when you start your Vibe Coding session, you don’t just prompt “make an app.” You provide the report:

"Gemini, using the identified 'Invisible Tracker' Vibe in ./docs/market-research/habit_vibe.md, scaffold a React + Vite frontend and a Go-based CLI tool. The CLI is the only way to log habits."

Because the AI now has the “Market Ground Truth,” it won’t waste time building notification settings or complex onboarding flows. It builds exactly what the crawl suggested was missing.

Deep Dive: Mastering the Advanced Features

To truly master the marketplace-report-crawler, you need to look beyond simple keyword searches. The tool allows for “Deep Sentiment Mapping” and “Competitor Velocity Tracking.”

Sentiment Mapping (The “Emotional” Crawl)

One of the most powerful aspects of the crawler is its ability to weigh reviews based on “Emotional Intensity.” A 1-star review that says “doesn’t work” is noise. A 3-star review that says “I’ve used this for 2 years but I’m finally leaving because they removed the CSV export” is gold.

The crawler uses LLM-based scoring to highlight “High-Retention Exiters”—users who were loyal but are now looking for an alternative. These are your first 100 customers. By targeting the specific features these users lost, your “Vibe” becomes a magnet for a ready-made audience.

Competitor Velocity

The crawler can also monitor the frequency of updates. If a top-rated app hasn’t been updated in 6 months, but its reviews are starting to skew negative due to “bugs on the new OS version,” you have found a Decaying Giant.

In Vibe Coding, you can outpace a Decaying Giant in a weekend. The marketplace-report-crawler identifies these targets by correlating “Update Timestamp” with “Sentiment Slope.” When the slope is negative and the velocity is zero, it’s time to strike.

Best Practices & Tips for High-Signal Crawling

To avoid getting banned by marketplaces and to ensure you aren’t drowning in low-quality data, follow these professional standards:

1. The “Recursive” Prompting Strategy

Don’t just crawl once. Use the output of your first crawl to refine your second. If the first report says “Users hate the pricing of App X,” run a second crawl specifically on “App X alternatives” to see what those competitors are doing right—or wrong—with their monetization.

2. Focus on “Recent” Friction

Markets move fast. A review from 2022 is irrelevant in 2026. Configure your crawler to prioritize data from the last 90 days. This ensures your Vibe is tuned to the current “Current OS” or “Current AI Trend” frustrations.

3. Cross-Marketplace Triangulation

A problem that exists on the Shopify App Store often exists in the WordPress Plugin repository in a different form. If you are building a B2B tool, crawl multiple platforms. If the “Sync with Slack” pain point appears in both ecosystems, you’ve found a universal problem that scales.

4. Respect the Robots (Ethically and Technically)

The crawler is powerful, but it must be used responsibly.

Rate Limiting: Always use the built-in adaptive-delay setting. Blasting a marketplace with 1,000 requests in a second will get your IP flagged and provide zero data.
Data Privacy: The crawler is designed to extract trends, not identities. Avoid storing personally identifiable information (PII) from reviews. Focus on the what, not the who.

Integrating with the “Cody Master” Workflow

Within the Cody Master ecosystem, the marketplace-report-crawler serves as the “Pre-Flight” phase of the development lifecycle.

Phase 1 (Discovery): Activate marketplace-report-crawler. Generate the MARKET_VIBE.md.
Phase 2 (Strategy): Use cm-brainstorm-idea (another skill) to cross-reference the Vibe Report with your technical constraints.
Phase 3 (Implementation): Feed the finalized “Strategy Doc” into your execution agents.

By following this sequence, you eliminate the “Vibe Drift”—the tendency for a project to get more complex and less relevant as implementation continues. Every time you hit a fork in the road during coding, your agent can refer back to the crawler’s report to make the “Market-First” decision.

Conclusion: Data is the New Vibe

We are moving past the era where “Vibe Coding” meant “guessing until it looks cool.” The next generation of elite developers uses AI not just to write code, but to interpret the world.

The marketplace-report-crawler is your primary tool for this interpretation. It allows you to step outside your own bubble and see the digital world through the eyes of thousands of frustrated users. When you code with that level of clarity, you aren’t just building an app; you are providing a missing piece of the internet.

Master the crawler. Find the gap. Code the vibe. Win the market.

Next Steps: Try running a crawl on the most popular “AI Code Assistant” extensions. What are users complaining about today? Use that data to tune your own Cody Master configuration. The cycle of improvement never ends.

Mastering marketplace-report-crawler: The Complete Guide