Backward-Facing Step (pitzDaily)
URANS validation of turbulent separated flow
Predicted reattachment at x/h=7.3 (experimental range: 6–8h). Time-averaged over 19.8 flow-throughs using PETSc CG+GAMG on GPU.
Key Results
The Problem
The backward-facing step is a canonical benchmark for turbulent separated flows. Flow separates at the step edge and reattaches downstream — predicting where it reattaches tests whether your turbulence model and numerics capture the recirculation zone correctly.
Setup: Re_h = 25,400, expansion ratio 2.0. The expected reattachment length from experiments (Armaly et al.) is 6–8 step heights downstream.
What I Did
1. Started with steady-state, hit a wall. I ran k-epsilon with simpleFoam first. The pressure residual stalled at 1.07e-2 and wouldn’t converge. Instead of assuming a numerical bug, I recognized this as a physical signal: the shear layer is inherently unsteady (Kelvin-Helmholtz instability), so a steady solver can’t converge.
2. Switched to transient URANS. Moved to pimpleFoam with k-omega SST. SST blends k-omega (accurate near walls) with k-epsilon (stable in the freestream), which matters for separated flows where the recirculation zone is wall-bounded.
3. GPU-accelerated pressure solver. Pressure is the bottleneck — it’s an elliptic system with the tightest coupling. I routed it through PETSc’s CG solver with GAMG preconditioner on the RTX 3050 Ti via CUDA, using petsc4Foam.
4. Time-averaged over 19.8 flow-throughs. After a 2.6 flow-through washout, I collected UMean, pMean, and UPrime2Mean statistics. The run was split into 9 thermal blocks (~78 min each) with cool-down periods in between — the laptop hits 99°C if you don’t manage thermals.
The Result
Reattachment at x/h = 7.3 — within the experimental range. The Reynolds stress profiles from UPrime2Mean show the expected peak in the shear layer emanating from the step edge.
Why This Was Hard on a Laptop
- 6 MPI ranks on P-cores only — the 8 E-cores at 3.5 GHz bottleneck MPI synchronization against P-cores at 4.6 GHz, so including them actually slows things down
- Memory-bandwidth bound — 6 ranks and 12 ranks give identical wall-clock time because they share the same 44.6 GB/s memory bus
- Thermal blocks — sustained runs of ~78 min, then wait 15–20 min for chassis ambient to drop below 45°C before restarting
Key Learnings
- simpleFoam phi field is incompatible with pimpleFoam — must delete and let pimpleFoam regenerate, otherwise Courant number explodes.
- On hybrid P/E-core laptops, 6 MPI ranks on P-cores matches 12-rank wall-clock time (memory-bandwidth bound) while staying thermally safe.
- Time-averaging endTime must align with writeInterval boundaries to capture the final snapshot.
- Chassis ambient temperature (not CPU die) is the real thermal gate — slow tau of ~582s means waiting 15–20 min between runs.