I think these days, most US schools teach it. They did to us in the '80s and '90s, though we originally planned to go metric until Regan I think killed it. We definitely were using it in the late '90s in all our science-related classes and I'm sure that's still true today.
The downside isn't the learning bit, it's that nothing else* uses it daily. I found the switch mostly painless (m/s for wind was a bit odd to me at first, but I like it at this point) and I think most people would be fine. I think older folks will have a hard time ballparking things in metric, at least for a while, as their minds are already using another system to estimate. That will fade, however.
A huge problem is all the signs and stuff that need to be replaced. The US is huge and has many, many streetsigns including in small areas that really don't have the money to do anything about it. On the whole in the longterm, going metric would have far more benefits than problems.
* technically, I think all US government does use it in many regards and all US Customary Units are pinned to metric, but it's not used by everyday people.