Knowledge
Batch your entire media library
Thousands of items? Processing runs in phases with progress shown in the dashboard.
Planning
Start outside peak hours; monitor server load on shared hosting.
How batch processing works under the hood
Spark doesn't process a media library all at once synchronously. Instead, it uses a queue that runs through in stages in the background. This way your website doesn't have to wait for one large request that could get stuck on hosting limits like maximum execution time or memory. You can see the progress in the dashboard, and if processing is interrupted—for example by a timeout or restart—it simply picks up where it left off.
Step-by-step guide: processing a large library in batches
Choose a quiet moment to start
Start a large batch preferably during off-peak hours, especially on shared hosting where resources are shared with other sites.
Begin with a small test set
First process a small batch or test set to check whether the chosen compression profile gives the desired result, before processing your entire library.
Start the full processing and monitor progress
Let the rest of the library process in stages and keep an eye on progress in the dashboard.
Monitor server load
Especially with shared hosting, it's wise to check during a large batch whether your site remains responsive.
Check the result and clear the cache
After completion, review a sample of processed files and clear your cache so visitors see the new, smaller files.
How long does a batch run take
Exact processing times vary greatly depending on your hosting environment, average file size, and your plan. Here's a rough estimate:
| Library Size | Estimated Processing Time | Points to Consider |
|---|---|---|
| Small (up to around 500 files) | Usually completed within a short timeframe. | Usually no special preparation needed. |
| Medium (around 500 to 5,000 files) | Typically processes in stages over a longer period. | Schedule during off-peak hours; check progress regularly. |
| Large (more than 5,000 files) | Processing can take days, depending on hosting and limits. | Consider a test set beforehand and account for your plan limit. |
Common mistakes with batch processing
- Expecting an entire library to be processed in one synchronous request; hosting limits for execution time or memory often make this impossible.
- Not accounting for shared hosting resources during a large batch.
- Making major manual changes to the media library while a batch is running.
- Not checking progress and assuming everything is working without reason.
- Not clearing the cache after completion, causing visitors to still see old files.
Frequently asked questions
Processing resumes where it left off. Files that are already done won't be processed again.
Yes, you can do it at the same time. New uploads are included according to your configured policy.
The processing runs in the background and is designed to minimize impact as much as possible. On heavily loaded shared hosting, a large batch can count toward total server load, which is why we recommend starting outside peak hours.
In the plugin dashboard you can see how many files have been processed and how many are still in the queue.
You'll get a notification around 80% and at 100% of your limit. After that, processing waits until your limit refreshes or you choose a more generous plan.
Yes, you can stop the processing and resume it later.