>"Sorry buddy, I wasn't really using the thing, I just added everyone, but thanks for letting me know you like me!"

Just say you were doing the same thing. Can atleast deny it to leave some doubt, can't really question why you added everyone to Secret Crush when they also apparently did it.