Designing Verifiable Rewards for Financial Agents

Designing Verifiable Rewards for Financial Agents

Financial environments fail when rewards look clean but hide accounting drift, tool misuse, or unverifiable assumptions. In practice, a robust reward function must validate both the final answer and the chain of quantitative operations behind it.

Start with Failure Taxonomy, Not Reward Math

Before tuning coefficients, catalog recurring failure modes: stale data usage, unit mismatch, unsupported assumptions, and offset drift in calculations. This taxonomy should drive your verifier architecture.

Separate Policy Score from Verification Score

Keep two channels:

• policy quality (strategy, coherence, prioritization)
• strict verification (numeric and procedural correctness)

Merging them too early often masks critical regressions.

Instrument Every Tool Call

For financial RLVR, tool traces are first-class signals. Capture source docs, transform steps, and calculation outputs so failed runs can be replayed deterministically and audited quickly.

Reward Convergence Requires Domain Reality

Synthetic tasks alone produce brittle gains. Pair model-generated examples with expert-authored scenarios from real analyst workflows to keep reward signals grounded.

Done well, the result is not just higher scores; it is a system you can trust under stress.