Referred aggression

I have two pairs who live separately and their runs are 90 degree angles to each other, one across the back fence and one down the side, and the closest corners of the runs are only approx 5 feet apart. They can see each other all day long with no issues. But when one pair free range I have to either confine the other pair to the shed, or temporarily box in the others run with plywood so they can't see the free ranging pair. If I don't we get scuffles from the pair in the run, or shed.
I understand this is referred aggression, and there is probably not a lot I can do to stop it, but how do those of you with lots of buns cope with this? It's like a military operation each time I want to let the buns free range and I'm always on tenderhooks.
Any ideas?

Yes, I find if they can't see the free ranging buns then they are happy. There is nothing you can do to stop them doing this, even a Mum will attack her young, it is most peculiar.

It generally seem to be them reacting to having movement so close to the boundary of their territory - although we know the other rabbits can't cross the 'fence' I don't think they quite get that concept so it's very stressful for them to see them there but not be able to chase them away - which is what their instincts say. Cutting off the sight line is really the only option - if the plywood is tricky then maybe get something lighter/easier to move. Even a pegged on sheet - for example should do the trick.

Of course there is one other option - you could attempt turning them into a group a four - how they react over boundaries isn't an indication of how they'll react in neutral territory so it may work

Thanks for replying guys. Yes I think a sheet or something I can peg on May be the answer as the wind sometimes blows the ply down and then it's panic stations... From me anyway!
Was seriously thinking of trying to bond them into a quad, but I'm now going to rescue another bun, so planning on having a pair and a trio. I think a 5 group bond may be more than I can handle!