'Work' can include things like travel and lunches. Both of which can be downtime but are classed as 'work'.

Overall its hard to believe considering he also has Space x and other investments and im suresomeone is streching the truth. But I dont doubt the guy lives and breaths his work.