No, it is not wrong at all. In the SICP video lectures they call it just Lisp all of the time.
It's true that when someone says Lisp they often mean CL. If it's important for you to highlight that you are talking about the family of languages, you may say "the Lisps" or "a Lisp". But just Lisp should be okay too.
Good to know. Yes I love the SICP videos and noticed that - but then they were done in the 80's, wasn't sure if things had changed somewhat since then.
In my limited exploration, I always though CL was showing its age, yet scheme seemed more "timeless" - if you can say that about programming languages.
It's true that when someone says Lisp they often mean CL. If it's important for you to highlight that you are talking about the family of languages, you may say "the Lisps" or "a Lisp". But just Lisp should be okay too.