Yandex proposes AI test for spiritual values
Yandex is preparing to test AI on 'spiritual and moral values' — Kommersant At a meeting held in August on regulating AI, Yandex presented to the authorities a concept of test tasks and benchmarks for checking AI compliance with 'spiritual and moral values'.
Consensus
- Yandex presented a concept for a national dataset for evaluating AI systems on August 13.
- The dataset aims to assess whether AI systems align with 'spiritual and moral values' and are suitable for use in Russia.
- The dataset is still in the conceptual stage.
- There is disagreement over whether the dataset should be public or closed.
- The FSB supports a closed format to prevent model manipulation.
- Business representatives warn a closed format could be overly complex and slow development.
- A compromise involving both open and closed formats is being discussed.
- The content of the dataset has not yet been determined.
- An interagency commission involving government, business, and experts is proposed to define the content.
- The project is under discussion with relevant authorities.
Points of divergence
- The FSB insists the dataset must be closed so developers cannot intentionally train models to pass the test. — meduza
- The proposed compromise includes an open benchmark for public testing and transparency, and a closed one for final verification with different question formulations. — thebell
- The project is still at the concept stage, and details such as public access and content determination remain unresolved. — tg_paperpaper_ru
- The project is still in the concept stage, and it remains unclear whether tests will be accessible or closed. — tg_tvrain
- The government confirmed that the project is still under discussion, and Yandex and the AI Alliance did not respond to inquiries. — tvrain
- The proposed compromise includes an open benchmark for transparency and a closed one for final testing with identical themes but different question wording. — kommersant
Coverage (6 sources)
- Yandex has come up with a way to evaluate AI in line with 'spiritual-moral values'. The FSB insists that this test should be closed — Meduza
- «Yandex» presented tests for national AI. Without human participation they will still not get around — The Bell
- «Yandex» is preparing to test AI on «spiritual and moral values» — «Kommersant» — Бумага
- Yandex presented first tests for checking AI against 'spiritual-moral values' — Дождь (Telegram)
- Yandex presented first tests for checking AI on compliance with spiritual-moral values — Дождь
- Yandex presented tests for AI to comply with spiritual and moral values — Коммерсантъ