I've heard the same, I also know he's a liar and drugs were known to be all over the White House when he was president. I can believe he doesn't drink, but his personality and behavior (e.g. working/tweeting hours) say cocaine to me.
I feel like if he were a user, he'd have been caught by now, and losing a brother (iirc) to addiction is a solid enough reason to expect sobriety of him.