Skip to content

Scheduled prune for hosted social media derivatives (all networks) #121

Description

@paulocastellano

Context

Some social networks pull media from a URL instead of accepting an uploaded byte stream (TikTok, Instagram, Threads, Facebook). For those, when we need a processed copy of an image (aspect-ratio crop, or a resized/spec-compliant version), we render a derivative, host it on our public disk, and hand the platform that URL.

Today there are two derivative directories:

Directory Written by Used for Cleanup today
social-crops/ App\Services\Social\Concerns\CropsImageForAspectRatio (:42) Instagram/Facebook aspect-ratio crops None — leaks forever
social-tiktok-photos/ App\Services\Social\TikTokPublisher::renderCompliantPhoto (:333) TikTok photos resized to ≤1080px (PR #120) Inline Storage::delete in a finally after publish

Neither approach is right long-term:

  1. social-crops/ leaks. Every Instagram/Facebook post with an aspect ratio writes a JPEG that is never deleted. These accumulate on disk indefinitely.
  2. social-tiktok-photos/ inline prune has a rare timing risk. The derivative is deleted in a finally right after waitForPublishStatus returns. On the normal PUBLISH_COMPLETE path (and on failure paths) TikTok has already pulled the image, so it's safe. But if the status poll times out (~60s) while TikTok is still pulling, the inline delete removes an image the platform still needs.

Proposal

Replace the per-publisher cleanup strategy with a single scheduled prune command that periodically deletes derivative files older than a safe window (e.g. 1 hour — far longer than any platform needs to pull) from all social-derivative directories.

This is intentionally generic so that every current and future social network that hosts a pull-from-URL derivative is covered by one mechanism.

Tasks

  • Create php artisan make:command PruneSocialMediaDerivatives (sibling of the existing scheduled commands: ProcessScheduledPosts, RefreshExpiringTokens, CheckSocialConnections).
    • Delete files older than a configurable age (default 1h) from the known derivative directories: social-crops/, social-tiktok-photos/.
    • Source the directory list from a single place (e.g. a shared constant/config) so adding a new network's directory is one line.
  • Register it in the scheduler (->hourly() or similar) in routes/console.php / bootstrap/app.php.
  • Remove the inline finally prune in TikTokPublisher::publishPhotos (:278) now that cleanup is centralized — this also removes the timeout risk above.
  • Tests: command deletes old files, keeps recent ones, and is a no-op when the directories are empty/missing.

Notes / future-proofing

  • When a new pull-from-URL platform starts hosting derivatives (e.g. if we ever generalize the TikTok resize to Instagram/Threads/Facebook), it must write into a directory registered with this prune, not invent an uncovered one.
  • Use the application default disk (never force a disk name), consistent with the existing publishers.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions