Pyra Eval Working Group

🎉 Pyra-XL 在内部 LMArena-Δ 上达到 ELO 1387。明天的 WG 我会复盘评测设置。

🎉 Pyra-XL hit ELO 1387 on internal LMArena-Δ. I'll walk through the eval setup at tomorrow's WG.

大家好——我注意到我们的 LMArena-Δ 评测 setup 跟 LMArena 公开 setup 有 11 处差异。我整理了 diff 表，传到 Files。

📎 LMArena_diff_v3.xlsx

Hey all — I noticed our LMArena-Δ eval setup diverges from the public LMArena setup in 11 places. I put a diff table together and uploaded it to Files.

📎 LMArena_diff_v3.xlsx

本帖已被管理员删除。

This post has been removed by an admin.

泄露的 LMArena diff（来自上周的 leadership preview deck）：

· Pyra-XL preference-optimized 版本上了 internal Bench #2
· Pyra-XL production checkpoint 在外部评测排第 32
提交到 LMArena 的是哪一个？谁负责的解释一下。我们不能"used different models for different benchmarks"，这话上次就出过事。

Leaked LMArena diff (from last week's leadership preview deck):

· The Pyra-XL preference-optimized variant landed on internal Bench #2
· The Pyra-XL production checkpoint placed #32 on the external eval
Which one did we actually submit to LMArena? Whoever owned this — please explain. We can't fall back on "used different models for different benchmarks" — that line blew up in our faces last time.