Less is More for Long Document Summary Evaluation by LLMs