Skip to content

feat: add Bright Data agents for web intelligence and real-time web data#2104

Closed
felixLandlord wants to merge 4 commits into
github:stagedfrom
felixLandlord:brightdata-agent
Closed

feat: add Bright Data agents for web intelligence and real-time web data#2104
felixLandlord wants to merge 4 commits into
github:stagedfrom
felixLandlord:brightdata-agent

Conversation

@felixLandlord

Copy link
Copy Markdown

Pull Request Checklist

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have read and followed the Guidance for submissions involving paid services.
  • My contribution adds a new instruction, prompt, agent, skill, workflow, or canvas extension file in the correct directory.
  • The file follows the required naming convention.
  • The content is clearly structured and follows the example format.
  • I have tested my instructions, prompt, agent, skill, workflow, or canvas extension with GitHub Copilot.
  • I have run npm start and verified that README.md is up to date.
  • I am targeting the staged branch for this pull request.

Description

This contribution adds two GitHub Copilot custom agents that connect Copilot to Bright Data's Web MCP server for live web intelligence:

  • brightdata-rapid-agent — Real-time web research on Bright Data's free tier (5,000 requests/month). Uses discover, search_engine, scrape_as_markdown, and batch variants for search and page retrieval.
  • brightdata-pro-agent — Full Bright Data MCP Pro mode with 60+ tools: everything in Rapid plus browser automation (scraping_browser_*), structured platform extractors (web_data_* for Amazon, LinkedIn, X, npm, etc.), and extract.

Both agents instruct Copilot to route all live web work exclusively through Bright Data MCP—not built-in web fetch or training data—and to cite tool output with title and URL. Users select the agent manually from the Copilot agent dropdown on github.com/copilot/agents, when assigning issues, in Copilot Chat /task, or via Copilot CLI (/agent or --agent=).

Setup: Add an Agents secret named COPILOT_MCP_BRIGHTDATA_TOKEN with a Bright Data API token. Requires a paid GitHub Copilot plan (Pro, Pro+, Business, or Enterprise). The Pro agent appends &pro=1 to the MCP URL for pay-as-you-go billing; Rapid uses the standard endpoint.


Type of Contribution

  • New instruction file.
  • New prompt file.
  • New agent file.
  • New plugin.
  • New skill file.
  • New agentic workflow.
  • New canvas extension.
  • Update to existing instruction, prompt, agent, plugin, skill, workflow, or canvas extension.
  • Other (please specify):

Additional Notes

Paid service disclosure: Bright Data is a third-party paid service. The Rapid agent works on the free tier (5,000 MCP requests/month); Pro mode is usage-based. Users must create their own Bright Data account and supply their own API token via COPILOT_MCP_BRIGHTDATA_TOKEN. No credentials are embedded in the agent files.

MCP configuration: Agent-level mcp-servers in the YAML is used by Copilot cloud agent on GitHub.com. Copilot CLI users must configure the Bright Data MCP server locally with /mcp add; repository MCP settings do not carry over to CLI.

Testing: Agents were tested on GitHub.com cloud agent and Copilot CLI with live search, scrape, and (Pro) browser automation tasks. Both agents include retry and error-handling instructions for common MCP failures (502, timeout, empty body).

Official references:


By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.

- Introduced `brightdata-pro-agent` for advanced web intelligence with 60+ tools for scraping and automation.
- Added `brightdata-rapid-agent` for real-time web research using the Bright Data MCP free tier.
- Updated documentation to include both agents in the agents' README.
Copilot AI review requested due to automatic review settings June 23, 2026 16:35
@github-actions github-actions Bot added agent PR touches agents new-submission PR adds at least one new contribution labels Jun 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🟡 Contributor Reputation Check: MEDIUM risk

Check Risk
Profile MEDIUM
Credential audit NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

@github-actions github-actions Bot added the needs-review:MEDIUM Contributor reputation check flagged MEDIUM risk label Jun 23, 2026
@github-actions

github-actions Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

🔍 Skill Validator Results

✅ All checks passed

Scope Checked
Skills 0
Agents 2
Total 2
Severity Count
❌ Errors 0
⚠️ Warnings 0
ℹ️ Advisories 0

Summary

Level Finding
ℹ️ Found 2 agent(s)
ℹ️ Validated 2 agent(s)
ℹ️ ✅ All checks passed (2 agent(s))
Full validator output
Found 2 agent(s)
Validated 2 agent(s)
✅ All checks passed (2 agent(s))

@github-actions

Copy link
Copy Markdown
Contributor

🔒 PR Risk Scan Results

Scanned 3 changed file(s).

Severity Count
🔴 High 0
🟠 Medium 0
ℹ️ Info 0

✅ No matching risk patterns were detected in changed files.

This is an automated soft-gate report. Findings indicate review targets and do not block merge by themselves.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds two new GitHub Copilot custom agents that configure and enforce using Bright Data’s MCP server for live web intelligence (Rapid/free tier vs Pro mode), and registers them in the generated agents documentation.

Changes:

  • Added a Rapid-mode Bright Data web research agent with a constrained tool workflow and failure-handling guidance.
  • Added a Pro-mode Bright Data web intelligence agent including guidance for web_data_*, extract, and scraping_browser_* tools.
  • Updated docs/README.agents.md to include the two new agents and MCP install links.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
docs/README.agents.md Adds the two Bright Data agents to the generated agents table and includes MCP install links.
agents/brightdata-rapid-agent.agent.md Introduces the Rapid (free-tier) Bright Data MCP-backed research agent definition and usage rules.
agents/brightdata-pro-agent.agent.md Introduces the Pro Bright Data MCP-backed web intelligence agent definition and usage rules.

Comment thread agents/brightdata-rapid-agent.agent.md
@@ -0,0 +1,72 @@
---
name: brightdata-pro-agent
Comment thread docs/README.agents.md
Comment on lines +62 to +63
| [Brightdata Pro Agent](../agents/brightdata-pro-agent.agent.md)<br />[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbrightdata-pro-agent.agent.md)<br />[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbrightdata-pro-agent.agent.md) | Web intelligence via Bright Data MCP Pro — 60+ tools for search, scrape, platform data, and browser automation | [brightdata](https://github.com/mcp/io.github.brightdata/brightdata-mcp)<br />[![Install MCP](https://img.shields.io/badge/Install-VS_Code-0098FF?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-vscode?name=brightdata&config=%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%26pro%3D1%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D)<br />[![Install MCP](https://img.shields.io/badge/Install-VS_Code_Insiders-24bfa5?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-vscodeinsiders?name=brightdata&config=%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%26pro%3D1%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D)<br />[![Install MCP](https://img.shields.io/badge/Install-Visual_Studio-C16FDE?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-visualstudio/mcp-install?%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%26pro%3D1%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D) |
| [Brightdata Rapid Agent](../agents/brightdata-rapid-agent.agent.md)<br />[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbrightdata-rapid-agent.agent.md)<br />[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://aka.ms/awesome-copilot/install/agent?url=vscode-insiders%3Achat-agent%2Finstall%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2Fgithub%2Fawesome-copilot%2Fmain%2Fagents%2Fbrightdata-rapid-agent.agent.md) | Real-time web research via Bright Data MCP (free tier) — search, discover, and scrape | [brightdata](https://github.com/mcp/io.github.brightdata/brightdata-mcp)<br />[![Install MCP](https://img.shields.io/badge/Install-VS_Code-0098FF?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-vscode?name=brightdata&config=%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D)<br />[![Install MCP](https://img.shields.io/badge/Install-VS_Code_Insiders-24bfa5?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-vscodeinsiders?name=brightdata&config=%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D)<br />[![Install MCP](https://img.shields.io/badge/Install-Visual_Studio-C16FDE?style=flat-square)](https://aka.ms/awesome-copilot/install/mcp-visualstudio/mcp-install?%7B%22url%22%3A%22https%3A%2F%2Fmcp.brightdata.com%2Fmcp%3Ftoken%3D%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%2C%22headers%22%3A%7B%22Authorization%22%3A%22Bearer%20%24%7B%7B%20secrets.COPILOT_MCP_BRIGHTDATA_TOKEN%20%7D%7D%22%7D%7D) |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

🟡 Contributor Reputation Check: MEDIUM risk

Check Risk
Profile MEDIUM
Credential audit NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

@github-actions github-actions Bot added the skill-check-error Skill validator reported errors label Jun 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🟡 Contributor Reputation Check: MEDIUM risk

Check Risk
Profile MEDIUM
Credential audit NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

@github-actions

Copy link
Copy Markdown
Contributor

🟡 Contributor Reputation Check: MEDIUM risk

Check Risk
Profile MEDIUM
Credential audit NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

@github-actions github-actions Bot removed the skill-check-error Skill validator reported errors label Jun 23, 2026
@aaronpowell

Copy link
Copy Markdown
Contributor

Upon review, this is a little too heavily reliant on a paid service to fit into awesome copilot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent PR touches agents needs-review:MEDIUM Contributor reputation check flagged MEDIUM risk new-submission PR adds at least one new contribution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants