There are many potential benefits to rolling to IRA. However there are also potential disadvantages. As mentioned above, rolling traditional 401k can be a limitation for backdoor Roth, with the pro rata rule. I expect that’s the most common disadvantage for rolling to IRA, particularly if you are a high earner. Other potential disadvantages include not being eligible for rule of 55, less legal protection of retirement funds, loss of 401k loan benefits, potential loss of tax benefits for NUA stock, loss of RMD flexibility, and loss of institutional level investment options.
Increased complexity including in tracking different accounts can also be a disadvantage, as you mention. There are many options for tracking investments in different brokerages, which list stats about your combined investments in the different brokerages as well as list the totals individually. This a relatively common feature of brokerages and spending trackers. I use Fidelity Full View for this purpose. However, I would not recommend Full View to others, as Fidelity plans on switching to a new Full View that removes their Investment tracking features next month, in spite of Fidelity users as a whole hating their repeated attempts to switch to the new Full View. In the past Fidelity has tried to switch 1-2x per year, then returned back to the original Full View after user complaints.