Several former British colonies are now wealthy, first-world nations e.g. Ireland, Australia and America. Being an empire at one period in history would explain why a country was much wealthier than others, but not anymore - now it's trade. Just look at the rise of China.

(Some might argue that America is a de facto empire today, but even then most of its wealth is due to trade).