Make sure you communicate your concerns to him clearly. Communicate specific things that you have a problem with (specific so that he has an opportunity to rectify the situation). Also communicate the importance of the problems. He may know that you are not satisfied with the designer and the company's progress, but does he know that it's so important to you that you're willing to throw in the towel?

If, on the other hand, you've already communicated specific problems and the importance of the problems and he hasn't shown a good faith effort in helping you resolve these concerns, then you may need to part ways or otherwise restructure the company.