The Right Tool for the Job: Fixing a Slow PHP Queue With a Small Go Program

TL;DR

A friend of mine had a queue with a huge number of jobs to process. His stack was PHP, and since PHP runs single-threaded by default, the whole queue was processed one item at a time. It was taking forever.

I didn't ask him to rewrite his app. I just wrote a small Go program that pulled from the same queue and fanned the work out across goroutines. Same data, same destination, different tool for that one job. The process that used to take a long time finished in about 3 minutes.

This isn't a "PHP is bad, Go is better" post. It's about noticing when the tool you already have isn't the right fit for the specific job in front of you, and being willing to reach for something else just for that one piece.

The Problem

My friend has been building things in PHP and Laravel for about 10 years. It's not a "just started out" stack for him, it's his whole career. His app was built in PHP, and it worked fine for what it was built for: handling requests, running business logic, talking to the database. Normal web app stuff.

But at some point he had to process a large queue, and PHP, run the usual way, processes things one at a time. Each item waits for the one before it to finish. When the queue is small, you never notice. When the queue gets big, that single-threaded nature turns into a wall.

He was watching a job crawl through the queue, checking the database every so often to confirm things were actually moving, and it clearly wasn't going to finish in a reasonable amount of time.

Why I Didn't Touch His PHP Code

The tempting move here is to start "fixing" PHP: add multiprocessing, spin up multiple workers, wrestle with pcntl or a job runner, restructure how the app dispatches jobs. All of that is possible, and for a lot of teams it's the right long-term answer.

But this wasn't a "redesign the architecture" problem. It was a "this one queue needs to drain faster, today" problem. Rewriting how his whole app handles concurrency just to solve one slow job would have been a lot of risk and effort for something that didn't need to live inside PHP at all.

So instead, I stepped outside his app entirely.

Enter a Small Go Program

I wrote a small standalone Go program with one job: read from the same queue, process each item, and write the result to the same destination his PHP app was already using. Nothing fancy, no framework, just a worker pool.

Something in the shape of:

func main() {
	jobs := make(chan Job, 100)
	var wg sync.WaitGroup

	workers := 20
	for i := 0; i < workers; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for job := range jobs {
				process(job)
			}
		}()
	}

	for _, job := range fetchQueue() {
		jobs <- job
	}
	close(jobs)

	wg.Wait()
}

That's it. Go's goroutines made spreading the work across many workers almost free to write, and the runtime handles the scheduling for you. No new infrastructure, no message broker swap, no changes to his PHP codebase. It just sat next to his app, did one job, and could be thrown away afterward.

The Result

The queue that used to crawl finished in about 3 minutes. My friend checked the database afterward to make sure everything landed correctly, and it did. Same data, same guarantees, just processed by something built for concurrent work instead of something processing one item at a time.

He was, understandably, a little shaken by the difference. Not because PHP is bad, but because the gap between "one at a time" and "twenty at a time" is enormous once a queue gets large enough.

What stood out to me is that this is someone who has known nothing but PHP and Laravel for a decade, and he didn't need to defend his stack or feel bad about it. He just said "gokss" and went back to work. He also didn't have to learn Go to benefit from it. I handed him a working process, and it slotted in next to the app he already knew how to run.

He joked afterward that maybe he should learn Go someday. Maybe he will, maybe he won't. Either way, that's a separate decision from what actually solved his problem that day.

The Real Lesson

The point of this story isn't "use Go instead of PHP." My friend didn't rewrite his app, and I didn't tell him to. His PHP app is still doing exactly what it was built to do, and it's doing it well.

The point is that reaching for a different tool for one specific, well-defined piece of work isn't a betrayal of your stack. It's just recognizing what the job actually needs. PHP being single-threaded by default isn't a flaw, it's a tradeoff that makes sense for request/response web work. It's just not a tradeoff that makes sense for fanning out a huge queue.

You don't need to migrate everything, pick a new favorite language, or start a framework war in your head. You need to notice when a specific problem has a shape that a different tool handles naturally, use that tool for that problem, and get back to work. The experience of knowing that option exists, and being comfortable reaching for it, is worth more than any single language choice.

That's it. Right tool, right job, back to shipping.