My understanding about Chinese is that what makes it hard for English speakers is that it’s a tonal language. Hearing the tones at all, and even more so reproducing them, can be very hard for many English native adults.