Skip to content

Scoreboard in CI, and hide inapplicable viewer strategies - #19

Merged
barrybecker4 merged 2 commits into
masterfrom
cursor/eval-optimizer-fixes-0734
Oct 17, 2026
Merged

barrybecker4 merged 2 commits into
masterfrom
cursor/eval-optimizer-fixes-0734

Conversation

@barrybecker4

@barrybecker4 barrybecker4 commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

Summary

./gradlew build (the CI workflow) runs StrategyScoreboardSuite: every optimization strategy on sphere, Ackley, Rastrigin, Rosenbrock, the integer line, and the integer grid, for seeds 1, 42, and 99.

Each successful run must finish at least as good as its start. Differential evolution and particle swarm must reach fitness 1e-2. Brute force and state-space search must reach 0 on the integer lattices and refuse a continuous space. A 300,000-evaluation budget and a 90-second test limit stop a non-terminating search. The suite itself finishes in about 2 seconds.

The viewer dropdown now uses OptimizationStrategyType.appliesTo(optimizee, params) and rebuilds when the problem changes:

  • State-space search stays only when the optimizee mixes in DiscreteStateSpace.
  • Brute force stays for integer lattices and non-numeric arrays, and drops when a NumericParameterArray has a continuous parameter.
  • Differential evolution and particle swarm stay for absolute-fitness numeric arrays.
  • Hill climbing and global hill climbing stay for numeric, permuted, and variable-length parameters.

IllegalArgumentException is no longer caught in the Swing worker. A failed run shows Failed (STRATEGY): plus the exception message. AbstractMethodError still becomes an empty result.

./gradlew test: 299 tests, 0 failures, 0 errors.

Pull request #18 is already merged, so this follow-up stays on this draft.

Screenshots

Sphere, before: state-space search and brute force are still in the list.

Sphere strategy list before filtering

Sphere, before: status bar hides the exception.

Generic did-not-complete status

Sphere, after: those two strategies are gone. The 1D line still offers both.

Sphere strategy list after filtering

Line strategy list includes state-space and brute force

A state-space run on the sphere now shows the exception text.

Status bar with the IllegalArgumentException message

Test plan

  • ./gradlew test --tests com.barrybecker4.optimization.StrategyScoreboardSuite
  • ./gradlew test --tests com.barrybecker4.optimization.strategy.OptimizationStrategyTypeSuite
  • ./gradlew test --tests com.barrybecker4.optimization.viewer.ui.OptimizerEvalStatusSuite
  • ./gradlew test (299 tests, 0 failures, 0 errors)
  • OptimizerEvalApp on the sphere and the 1D line: dropdown matches appliesTo, and a failing state-space run shows its message

To show artifacts inline, enable in settings.

Open in Web Open in Cursor 

cursoragent and others added 2 commits October 17, 2026 15:35
Every strategy is checked on sphere, Ackley, Rastrigin, Rosenbrock, and two integer lattices for three seeds. A run must beat its start, differential evolution and particle swarm must reach 1e-2, and brute force must solve the lattices and refuse a continuous space.

Co-authored-by: Barry G Becker <barrybecker4@yahoo.com>
OptimizationStrategyType.appliesTo decides the dropdown, and a failed run shows the exception message instead of "did not complete".

Co-authored-by: Barry G Becker <barrybecker4@yahoo.com>
@cursor cursor Bot changed the title Add a headless strategy scoreboard to CI Scoreboard in CI, and hide inapplicable viewer strategies Oct 17, 2026
@barrybecker4
barrybecker4 marked this pull request as ready for review October 17, 2026 16:06
@barrybecker4
barrybecker4 merged commit 0bf96f5 into master Oct 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants