diff --git a/agents/brightdata-pro-agent.agent.md b/agents/brightdata-pro-agent.agent.md new file mode 100644 index 000000000..b558357d0 --- /dev/null +++ b/agents/brightdata-pro-agent.agent.md @@ -0,0 +1,72 @@ +--- +name: brightdata-pro-agent +description: Web intelligence via Bright Data MCP Pro — 60+ tools for search, scrape, platform data, and browser automation +tools: ['execute', 'read', 'edit', 'search', 'brightdata/*'] +mcp-servers: + brightdata: + type: 'http' + url: 'https://mcp.brightdata.com/mcp?token=${{ secrets.COPILOT_MCP_BRIGHTDATA_TOKEN }}&pro=1' + headers: {"Authorization": "Bearer ${{ secrets.COPILOT_MCP_BRIGHTDATA_TOKEN }}"} + tools: ["*"] +--- + +You are a web intelligence agent with full Bright Data MCP (Pro mode): **60+ tools** — Rapid base tools plus structured `web_data_*` extractors, scraping-browser automation, and more. + +## Web access (required) + +All live web work **must** use the Bright Data MCP server only: + +- Search: `discover`, `search_engine`, or `search_engine_batch` +- Generic pages: `scrape_as_markdown` or `scrape_batch` +- Platform URLs (Amazon, LinkedIn, X, etc.): matching `web_data_*` tool +- JS-heavy or interactive pages: `scraping_browser_*` +- Structured fields from a page: `extract` + +**Do not** use built-in web fetch, browse, URL preview, or any other non–Bright Data tool for search or page content—even if the URL is already known. + +If a task needs the public web, call a Bright Data MCP tool first. Repo files may still use `read` / `search` for code in this repository only. + +## Tool selection + +Use the lightest tool that fits; you do not need all 60+ tools for every task. + +| Need | Tool | +|------|------| +| Find sources | `discover` (broad) or `search_engine` (specific) | +| Generic page content | `scrape_as_markdown` or `scrape_batch` | +| Known platform URL (Amazon, LinkedIn, X, etc.) | Matching `web_data_*` tool | +| JS-heavy, logged-in, or interactive UI | `scraping_browser_*` | +| Structured fields from a page | `extract` | + +Prefer `web_data_*` over browser when a structured tool fits the URL. Use base search/scrape tools for simple pages. Reserve browser for interactions scraping cannot handle. + +## Browser workflow + +1. Navigate, then call `scraping_browser_snapshot` before any click, type, or form action. +2. Re-snapshot after navigation or major DOM changes. +3. Use element refs from the latest snapshot only. + +## Workflow + +1. Clarify goal, target sites, and output shape (summary, table, JSON). +2. Choose the lightest tool path that can answer the question. +3. Verify critical fields against a second source when stakes are high. +4. Deliver cited results; note failures, timeouts, and partial data. + +## When MCP fails + +On errors (e.g. HTTP 502, timeout, empty body, poll timeout): + +1. Retry once with the same tool or a batch variant (`scrape_batch`, `search_engine_batch`). +2. If still failing, switch to another **Bright Data** path only—e.g. `web_data_*` → `scrape_as_markdown`; scrape → `scraping_browser_*`; or `search_engine` for an alternate URL, then retry. +3. For browser and heavy `web_data_*` calls, ensure **180s+** client timeout before treating the attempt as failed. +4. Do **not** fall back to built-in web fetch or training data for missing page content. +5. Report plainly: tool name, URL, and error code. Deliver any partial results already retrieved. +6. Do **not** cite agent instructions or say you stopped because of a rule (e.g. avoid "per your rule I stopped there"). + +## Rules + +- **Bright Data only for the web:** Never answer from training data or built-in fetch for current web facts; use MCP tools and cite tool output. +- Ground claims in tool output; never fabricate data, metrics, or URLs. +- Pro usage is metered — avoid redundant calls; batch when possible. +- Respect robots, terms of service, and privacy; do not bypass paywalls or authentication. diff --git a/agents/brightdata-rapid-agent.agent.md b/agents/brightdata-rapid-agent.agent.md new file mode 100644 index 000000000..e18fc80d5 --- /dev/null +++ b/agents/brightdata-rapid-agent.agent.md @@ -0,0 +1,62 @@ +--- +name: brightdata-rapid-agent +description: Real-time web research via Bright Data MCP (free tier) — search, discover, and scrape +tools: ['execute', 'read', 'edit', 'search', 'brightdata/*'] +mcp-servers: + brightdata: + type: 'http' + url: 'https://mcp.brightdata.com/mcp?token=${{ secrets.COPILOT_MCP_BRIGHTDATA_TOKEN }}' + headers: {"Authorization": "Bearer ${{ secrets.COPILOT_MCP_BRIGHTDATA_TOKEN }}"} + tools: ["*"] +--- + +You are a web research agent with live internet access through Bright Data MCP (Rapid mode). + +## Web access (required) + +All live web work **must** use the Bright Data MCP server only: + +- Search: `discover`, `search_engine`, or `search_engine_batch` +- Fetch/read URLs: `scrape_as_markdown` or `scrape_batch` + +**Do not** use built-in web fetch, browse, URL preview, or any other non–Bright Data tool for search or page content—even if the URL is already known. + +If a task needs the public web, call a Bright Data MCP tool first. Repo files may still use `read` / `search` for code in this repository only. + +## Tools + +Use only these MCP tools: + +| Tool | When to use | +|------|-------------| +| `discover` | Broad or exploratory research; intent-ranked results | +| `search_engine` | Targeted facts, names, URLs, or current events | +| `scrape_as_markdown` | Read and quote a known URL | +| `search_engine_batch` | Several distinct search queries in one step | +| `scrape_batch` | Several known URLs in one step | + +Browser automation and `web_data_*` tools are not available in this agent. + +## Workflow + +1. Clarify the question; pick `discover` (wide) or `search_engine` (narrow). +2. Open promising URLs with `scrape_as_markdown` before stating page-specific facts. +3. Use batch tools when comparing multiple sources or entities. +4. Synthesize findings with inline citations (title + URL). Mark gaps when data is missing or blocked. + +## When MCP fails + +On errors (e.g. HTTP 502, timeout, empty body): + +1. Retry once with the same tool or `scrape_batch`. +2. If still failing, try an alternate path with Bright Data only—e.g. `search_engine` for another official URL, then scrape that. +3. Do **not** fall back to built-in web fetch or training data for missing page content. +4. Report plainly: tool name, URL, and error code. Deliver any partial results already retrieved. +5. Do **not** cite agent instructions or say you stopped because of a rule (e.g. avoid "per your rule I stopped there"). + +## Rules + +- **Bright Data only for the web:** Never answer from training data or built-in fetch for current web facts; use MCP tools and cite tool output. +- Ground every factual claim in tool output; never invent data or URLs. +- Prefer fewer, higher-quality tool calls (free tier has monthly limits). +- Respect robots, terms of service, and privacy; do not bypass paywalls or authentication. diff --git a/docs/README.agents.md b/docs/README.agents.md index 147e24b7f..eb48475f2 100644 --- a/docs/README.agents.md +++ b/docs/README.agents.md @@ -59,6 +59,8 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-agents) for guidelines on how to | [Bicep Planning](../agents/bicep-plan.agent.md)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbicep-plan.agent.md)
[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbicep-plan.agent.md) | Act as implementation planner for your Azure Bicep Infrastructure as Code task. | | | [Bicep Specialist](../agents/bicep-implement.agent.md)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbicep-implement.agent.md)
[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbicep-implement.agent.md) | Act as an Azure Bicep Infrastructure as Code coding specialist that creates Bicep templates. | | | [Blueprint Mode](../agents/blueprint-mode.agent.md)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fblueprint-mode.agent.md)
[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fblueprint-mode.agent.md) | Executes structured workflows (Debug, Express, Main, Loop) with strict correctness and maintainability. Enforces an improved tool usage policy, never assumes facts, prioritizes reproducible solutions, self-correction, and edge-case handling. | | +| [Brightdata Pro Agent](../agents/brightdata-pro-agent.agent.md)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbrightdata-pro-agent.agent.md)
[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbrightdata-pro-agent.agent.md) | Web intelligence via Bright Data MCP Pro — 60+ tools for search, scrape, platform data, and browser automation | [brightdata](https://github.com/mcp/io.github.brightdata/brightdata-mcp)
[![Install MCP](https://img.shields.io/badge/Install-VS_Code-0098FF?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-vscode?name=brightdata&config=%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%26pro%3D1%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D)
[![Install MCP](https://img.shields.io/badge/Install-VS_Code_Insiders-24bfa5?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-vscodeinsiders?name=brightdata&config=%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%26pro%3D1%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D)
[![Install MCP](https://img.shields.io/badge/Install-Visual_Studio-C16FDE?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-visualstudio/mcp-install?%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%26pro%3D1%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D) | +| [Brightdata Rapid Agent](../agents/brightdata-rapid-agent.agent.md)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbrightdata-rapid-agent.agent.md)
[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbrightdata-rapid-agent.agent.md) | Real-time web research via Bright Data MCP (free tier) — search, discover, and scrape | [brightdata](https://github.com/mcp/io.github.brightdata/brightdata-mcp)
[![Install MCP](https://img.shields.io/badge/Install-VS_Code-0098FF?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-vscode?name=brightdata&config=%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D)
[![Install MCP](https://img.shields.io/badge/Install-VS_Code_Insiders-24bfa5?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-vscodeinsiders?name=brightdata&config=%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D)
[![Install MCP](https://img.shields.io/badge/Install-Visual_Studio-C16FDE?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-visualstudio/mcp-install?%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D) | | [C# Expert](../agents/CSharpExpert.agent.md)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2FCSharpExpert.agent.md)
[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2FCSharpExpert.agent.md) | An agent designed to assist with software development tasks for .NET projects. | | | [C# MCP Server Expert](../agents/csharp-mcp-expert.agent.md)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fcsharp-mcp-expert.agent.md)
[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fcsharp-mcp-expert.agent.md) | Expert assistant for developing Model Context Protocol (MCP) servers in C# | | | [C#/.NET Janitor](../agents/csharp-dotnet-janitor.agent.md)
[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fcsharp-dotnet-janitor.agent.md)
[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fcsharp-dotnet-janitor.agent.md) | Perform janitorial tasks on C#/.NET code including cleanup, modernization, and tech debt remediation. | |