Vision Language Model Guided Zero-shot Classification