Describe the problem
During the feed import process, the URL validity must be checked in both the creation and update of a feed. In the case of a schedule feed, the URL should produce a downloadable zip folder. In the case of a realtime feed, the URL should produce a protobuf stream.
There is one exception to this: occasionally we want to import a feed despite it not producing a downloadable zip because we still want to include it in the MobilityDatabase. In this scenario, we can forgo the URL validity check altogether because we aren't concerned about the result.
Proposed solution
We can perform a HEAD request on the URL to check the Content-Type and Content-Disposition. Schedule feeds should contain Content-Type: application/zip or application/octet-stream while realtime feeds should contain Content-Type: application/protobuf, application/x-protobuf, or application/vnd.google.protobuf.
We can consider including this in the web-app as something that is executed immediately upon submission. Then the URL validity can be directly included in the Github issue that is created.
Alternatives you've considered
No response
Additional context
In the event where a feed requires authentication, a person would have to fill out a token/key requisition before proceeding with the URL validity check.
Describe the problem
During the feed import process, the URL validity must be checked in both the creation and update of a feed. In the case of a schedule feed, the URL should produce a downloadable zip folder. In the case of a realtime feed, the URL should produce a protobuf stream.
There is one exception to this: occasionally we want to import a feed despite it not producing a downloadable zip because we still want to include it in the MobilityDatabase. In this scenario, we can forgo the URL validity check altogether because we aren't concerned about the result.
Proposed solution
We can perform a HEAD request on the URL to check the Content-Type and Content-Disposition. Schedule feeds should contain Content-Type: application/zip or application/octet-stream while realtime feeds should contain Content-Type: application/protobuf, application/x-protobuf, or application/vnd.google.protobuf.
We can consider including this in the web-app as something that is executed immediately upon submission. Then the URL validity can be directly included in the Github issue that is created.
Alternatives you've considered
No response
Additional context
In the event where a feed requires authentication, a person would have to fill out a token/key requisition before proceeding with the URL validity check.