Chatfield, gun policy, and the turn away from Nelson

Did machinery improvements chiefly cause the Royal Navy to abandon the G3 and Nelson-style layout?

No. Chatfield's traditionalism and preference for 14-inch volume of fire were more important.

In this account, Admiral Ernle Chatfield was a traditionalist committed to the 14-inch gun and volume of fire. That principle can outweigh calibre when comparing six-inch and eight-inch cruisers. It is much weaker when the choice is nine 16-inch guns versus ten 14-inch guns, because the barrel-count gain is small while each hit is lighter. The change from the Nelson lineage therefore reflects leadership and armament policy as well as technical design.