PersLitEval: Fine-grained Benchmark and Evaluation of LLMs on Persian Literature Questions

Despite impressive multilingual capabilities, large language models (LLMs) remain poorly evaluated on literary knowledge in non-English languages. We introduce PersLitEval, a benchmark of 4,514 Persian literature multiple-choice questions across eight fine-grained categories spanning spelling, liter...