December 30, 2025
Threads on threads on drama
Graph Algorithms in Rayon
Rust devs clash over “go fast” tricks, deadlocks, and how to tame chaos
TLDR: A Rust linker team tried multiple ways to speed up unpredictable graph work with Rayon; their channel + par_bridge trick reduced memory use but risks deadlocks and doesn’t play nice with other parallel bits. Commenters split between “use Rayon scopes/work stealing” and “stop reinventing schedulers,” with jokes and heat.
The Wild linker team dropped a behind-the-scenes peek at how they try to make graph-walking go faster with Rayon—a Rust tool for parallel work—and the comments immediately turned into a speedrun of opinions. The gist: when you don’t know how much work is coming (imagine a to‑do list that keeps growing), you need clever tricks to spread tasks across threads. Their journey: custom thread juggling, scoped tasks that felt pricey, and a new “channel + par_bridge” pipeline that reduced memory churn but flirted with deadlocks.
The community split fast. One camp cheered the channel idea as a gutsy hack, but the other camp shouted “you’re re‑inventing a scheduler” and begged them to stick with Rayon’s scope—even if it allocates more. The hottest take: the deadlock explained here (where a worker holds the sender and waits on the receiver) isn’t a “Rayon problem,” it’s an “architecture problem.” Meanwhile, the async crowd wandered in with “just use Tokio,” and got roasted because async is better for waiting on I/O, not crunching CPU.
Memes flew: “parked threads are on union break,” “deadlock is just an overly attached sender,” and a spicy chorus of “work stealing or bust.” Links to Rayon and Crossbeam popped up, but the mood was clear: speed is cool, composability is king, and deadlocks are the villain everyone loves to dunk on.
Key Points
- •The Wild linker needs parallel graph exploration where the total work is unknown, starting from root nodes and discovering more during traversal.
- •Using Rayon’s spawn_broadcast with custom job control is complex and prevents concurrent use of Rayon features like par_iter.
- •Scoped spawning with rayon::scope/in_place_scope works but is more expensive due to heap-allocated tasks, increasing allocations.
- •A crossbeam_channel + par_bridge approach reduces heap allocations and simplifies control but risks deadlocks when nested par_iter steals work while holding a sender.
- •The channel + par_bridge approach also suffers from poor composability with other Rayon operations, as noted by the author.