On Efficient Planning In Large Action Spaces With Applications To Cooperative Multi-Agent Reinforcement Learning