The factual accuracy of responses generated by large vision language models (LVLMs) is a key metric for assessing model capability, reflecting both the model's knowledge capacity and its reliability. To test the abilities to ``see the world and discover knowledge'' of LVLMs, we introduce the first knowledge-based visual question-answering Chinese benchmark, Chinese SimpleVQA. Key features of Chinese SimpleVQA include Chinese language focus, diverse knowledge types, a multi-hop question construction mechanism, high-quality data, static consistency, and easy-to-evaluate through short answers. It provides image object recognition questions alongside further knowledge-based questions, allowing users to explore the model's knowledge boundaries and analyze its performance at various stages. Our ultimate goal is to assist developers in better understanding and analyzing the factual capabilities of their models, thereby advancing the development of LVLMs and improving the factual accuracy of LVLMs in real-world applications.

Performance comparison of closed source and open source LVLMs on multi-hop QAs (i.e. Merged Q&A and Recognition Q&A). For metrics, CO, NA, IN, and CGA denote ``Correct'', ``Not attempted'', ``Incorrect'', and ``Correct given attempted'', respectively. The highest scores among models in each section are highlighted in green.