Smaller diameter has a thicker puck for the same weight of coffee.
Smaller diameter bottom doesnt bent as much under pressure like 58mm making the extraction more even from the center to the edges. When the basket bends under 7-12 bar the center is pushed out (watch videos of naked pf) and the water is forced to flow towards the middle, underextracting the outside edge (you see a black ring often at 58mm baskets).
Smaller diameter is also easier to extract with lower doses, for eg 14-15g. With 58mm it’s almost impossible with light stuff, maybe with turbo shots or paper filters yes.
I do not see any disadvantage of 54mm baskets, tampers are plenty now - bravo tamper, force tamper, stock tampers from Dalla Corte/LSM and so on. WDT tools are not basket size dependent, levelers/ocd tools are useless so why are you concerned about 54mm?>
Smaller diameter also holds higher pressure values better, since the thickness of the puck. 11 bar on 58mm is a no go.