feat: add FileDownloadCrawler

[Mantisus]

Description

• Adds FileDownloadCrawler with support for streaming the response body in the request handler.

Issues

• Closes: Implement a FileDownload crawler #875

Testing

• Added tests for FileDownloadCrawler

[Pijukatel]

For large files, it would be great to have something like https://pypi.org/project/smart-open/ , but our storages lacks interface to support such chunked uploads

[vdusek]

Otherwise it looks great

[vdusek]

From Claude:

The streamed path never sets context.request.loaded_url, whereas the buffered crawl() path does (request.loaded_url = str(response.url)). As a result, in stream mode _check_url_after_redirects is effectively skipped (it's guarded by if loaded_url is not None), so off-domain redirects aren't filtered by the enqueue strategy, and context.request.loaded_url stays None for user code. It's a silent divergence between the two modes. The stream() API doesn't expose the final URL, so this may be hard to fix cleanly, but worth a note/follow-up.

[Mantisus]

That's true. But we can't do that without updating the HTTP clients API. This will be fixed along with #874 when the clients are synchronized with Crawlee JS v4.