>>910754My buddy also had the problem of the brake rubbing. He has an aluminium cross too but a Ridley. For both of us it was only occasional but definitely down to torsion caused by weight. How close to the rotor do you keep your pads? I like for the brake to engage pretty early.
The twisting action would have to come from the weight of the panniers flexing the fork (and brake caliper) ever so slightly on the upper attachment points while it's all rigid down at the hub.
I should probably get another saddle as mine seems just slightly too narrow for my assbones. I tried DIYing my bikefit based off a few online guides. Occasional discomfort/mild pain in different places resulted, could probably get it much better if done professionally but that's almost 300 bucks here. Might have to for future long tours.
Another tip for OP. Make sure you have mounting holes for front racks halfway up the fork too since using those "p-clamps" is a really shitty solution there.