The intention of 967b6be118 ("ar8327: Add workarounds for AR8337
switch") was to remove the register fixups for AR8337. But instead they
were removed for AR8327.
The RGMII RX delay is forced even if the port is used as phy instead of
mac, which results in no package flow at least for one board.
Fixes: FS#1664
Signed-off-by: Mathias Kresin <dev@kresin.me>