PHP generators: process a feed without building a second giant array

An importer reads a large file, converts every record into an array, then loops over that array to save it. The temporary copy can become larger than the original file. A generator lets the consumer request one item at a time.

Start with a small executable example

<?php
function productCodes(int $count): Generator
{
    for ($i = 1; $i <= $count; $i++) {
        yield sprintf('PART-%06d', $i);
    }
}
$count = 0;
$first = null;
foreach (productCodes(10000) as $code) {
    $first ??= $code;
    $count++;
}
assert($first === 'PART-000001');
assert($count === 10000);
echo $count, PHP_EOL;

This example demonstrates consumption without retaining all codes. It is not a benchmark for your import or a promise of a particular memory reduction.

Keep the pipeline lazy

A generator does not help if the next line converts it into an array. The database layer can also undo the benefit by retaining every model, diagnostic payload, or result. Store bounded error samples and aggregate counts instead of an unbounded list of successful rows.

For a file reader, place fclose in a finally block around the yielding loop. That makes the resource lifetime explicit. If the caller retains a partially consumed generator, it also retains its local state until the generator finishes or is released.

Plan restart behavior separately

Streaming does not provide resumability. A process can still stop halfway through. For durable imports, record a feed identifier and an item-level external key. Design writes so replaying a processed record has a defined effect. Do not use a line number alone as identity if the supplier can reorder the file.

Measure peak memory on representative input and separately measure database duration. A lower memory peak can coexist with slow per-row queries. The goal is to identify the current bottleneck, not assume that yield solves all three problems. PHP's generator overview explains the iteration model.