Skip to content

Add the github_project_v2 table#548

Open
dark-panda wants to merge 1 commit into
turbot:mainfrom
dark-panda:add-project-v2-table
Open

Add the github_project_v2 table#548
dark-panda wants to merge 1 commit into
turbot:mainfrom
dark-panda:add-project-v2-table

Conversation

@dark-panda

Copy link
Copy Markdown

This PR adds a table called github_project_v2 that contains some of the fields exposed by the GraphQL API for ProjectV2 and adds a few fields to github_issue to allow you to join the two tables together. We're using the github_project_v2 name here rather than github_project because of existing fields in tables like github_team and github_organization that reference projects_v2_total_count and such.

Example queries:

=> select * from github_project_v2 where organization = 'turbot' ;
-[ RECORD 1 ]------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------
login_id                 | MDQ6VXNlcjg0Nzgz
organization             | turbot
number                   | 14
id                       | 10217986
node_id                  | PVT_kwDOAlEJmM4Am-oC
owner                    | {"type": "Organization", "user": {"id": 38865304, "login": "turbot"}, "organization": {"id": 38865304, "login": "turbot"}}
creator                  | {"url": "https://github.com/apps/github-project-automation", "login": "github-project-automation", "avatar_url": "https://avatars.githubusercontent.com/in/235829?v=4"}
title                    | Issue Management
description              |
is_public                | t
closed_at                | 2026-06-08 13:57:12+00
created_at               | 2024-08-29 03:57:44+00
updated_at               | 2026-06-08 13:57:12+00
state                    | closed
latest_status_update     | [NULL]
is_template              | f
readme                   | Tracking all examples required by end users
resource_path            | /orgs/turbot/projects/14
url                      | https://github.com/orgs/turbot/projects/14
repositories             | ["turbot/guardrails-samples"]
repositories_total_count | 1
teams                    | ["", ""]
teams_total_count        | 2
sp_connection_name       | github
sp_ctx                   | {"steampipe": {"sdk_version": "5.14.0"}, "connection_name": "github"}
_ctx                     | {"steampipe": {"sdk_version": "5.14.0"}, "connection_name": "github"}

=> select
  i.id,
  i.title,
  p.title as project_title,
  p.state as project_state,
  p.id as project_id
from
  github_issue i,
  jsonb_array_elements_text(i.project_items) as project_node_id
  join github_project_v2 p on p.node_id = project_node_id
where
  i.repository_full_name = 'turbot/guardrails-samples'
  and p.organization = 'turbot';
     id     |                           title                            |  project_title   | project_state | project_id
------------+------------------------------------------------------------+------------------+---------------+------------
  815566711 | SecHub - Feature - SecHub needs a heartbeat                | Issue Management | closed        |   10217986
  841771067 | GraphQL - Get the details of a policy such as descriptions | Issue Management | closed        |   10217986
  966425300 | Calculated Policy - Log Group Deletions                    | Issue Management | closed        |   10217986
 1016873038 | TrimBot - Fix the version for dependent libraries          | Issue Management | closed        |   10217986
  966422959 | TrimBot - Error handling should be more verbose            | Issue Management | closed        |   10217986
  966420924 | TrimBot - SNS topics not all removed                       | Issue Management | closed        |   10217986
(6 rows)

@cbruno10 cbruno10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dark-panda Thanks for raising this PR! I've added a few suggestions and questions in this PR. A few other minor points:

  • Can you please add a table doc page for docs/tables/github_project_v2.md?
  • Can you please also update the PR description, which have different (possibly older) columns? I think it'd be helpful to see the results of the example queries from the upcoming table doc.
  • Locally, models/project_v2.go and table_github_project_v2.go both fail gofmt -l, can you please check these files?

If you have any questions, please let us know!

{
Name: "updated_at",
Require: plugin.Optional,
Operators: []string{">", ">="},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the List function use the updated_at qual? Looking at the GraphQL projectsV2 field, I don't see a way to filter on the updated at time either, so not sure if we can filter on this field at the API layer.


func sharedProjectV2Columns() []*plugin.Column {
return []*plugin.Column{
{Name: "number", Type: proto.ColumnType_INT, Transform: transform.FromField("Number", "Node.Number"), Description: "The project number."},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the API ever return the data under Node.Number?

ShouldIgnoreError: isNotFoundError([]string{"404"}),
Hydrate: tableGitHubProjectV2List,
},
Get: &plugin.GetConfig{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the GetConfig work correctly when passing in organization and number?

KeyColumns requires organization and number, but the hydrate reads quals["id"] (never populated, so it's 0) and queries projectV2(id: $projectId). Looking at the GraphQL schema, Organization.projectV2 only accepts number, so this errors server-side with Unknown argument 'id'.

I think the Get function should use quals["number"] and use graphql:"projectV2(number: $number)" instead.

{Name: "user_did_author", Type: proto.ColumnType_BOOL, Hydrate: issueHydrateUserDidAuthor, Transform: transform.FromValue(), Description: "If true, user authored the issue."},
{Name: "user_subscription", Type: proto.ColumnType_STRING, Hydrate: issueHydrateUserSubscription, Transform: transform.FromValue(), Description: "Subscription state of the user to the issue."},
{Name: "assignees", Type: proto.ColumnType_JSON, Hydrate: issueHydrateAssignees, Transform: transform.FromValue().NullIfZero(), Description: "A list of Users assigned to the issue."},
{Name: "project_items", Type: proto.ColumnType_JSON, Hydrate: issueHydrateProjectItems, Transform: transform.FromValue().NullIfZero(), Description: "A list of project node IDs (PVT_...) for ProjectV2 projects the issue belongs to."},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{Name: "project_items", Type: proto.ColumnType_JSON, Hydrate: issueHydrateProjectItems, Transform: transform.FromValue().NullIfZero(), Description: "A list of project node IDs (PVT_...) for ProjectV2 projects the issue belongs to."},
{Name: "projects_v2", Type: proto.ColumnType_JSON, Hydrate: issueHydrateProjectItems, Transform: transform.FromValue().NullIfZero(), Description: "A list of ProjectV2 projects associated with the issue."},

For this column, similar to assignees and labels, I think we should return more project data, including its scalar fields as part of the project object. So we can reuse the ProjectV2 model you already defined and include its scalar fields:

  • id
  • node_id
  • number
  • title
  • description
  • state
  • is_public
  • created_at
  • updated_at
  • closed_at

Notably, we should skip the pageable connections (items/fields/views), and @include-gate anything heavy like latest_status_update. Users can still join on node_id, but now get titles/state without one.

Can you please also add a projects_v2_total_count column with the description Count of ProjectV2 projects associated with the issue.? The model already fetches ProjectItems.TotalCount, so it's basically free, and it matches the existing projects_v2_total_count columns in the github_organization and github_team tables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants