Outrider scans for promising ideas that fit your repository and automatically drafts code and tests. In PEFT, that work led to two merged contributions featured in v0.21.0.
The upstream contributions were evaluated manually. The Validate example is a new run on a SAELens pull request.
The method already had maintainer interest. Outrider surfaced it and drafted a fresh implementation. After human preparation and maintainer review, PR #3382 merged 33 days after opening.
Outrider identified a relevant idea and automatically drafted the implementation and tests. At first review, a PEFT maintainer wrote:
“this PR is already quite mature”
Benjamin Bossan · PEFT maintainer · first review of #3382 ↗ Nine targeted tests. The recorded benchmark remained below baseline LoRA in this configuration.
PR #1807 already contained maintainer reviews and requests to finish the work.
A broad draft stopped at preflight. A narrower implementation went forward for review.
A narrower PR followed author coordination in issue #3380.
Merged after five commits and two approvals. Benchmarks stayed public.
Different methods. Different maintainer standards. Every result is linked.
Accepted code contributions. See the review evidence ↓
Ask on the PR. Review the test. Validate runs the comparison and brings the evidence back.
| Measure | TopK baseline | AbsTopK change |
|---|---|---|
| Explained variance | 0.9666 | 0.9677 |
| Negative activations | 0% | 49.2% |
Allowed explained-variance loss: 0.02. Required negative activations: at least 1%. Both criteria passed in this run.
Remyx prepared the environment, ran the approved comparison and returned the notebook and metrics.
View the run timeline ↓Use the evidence to decide what to pursue, revise, or skip.
Criteria at the measured revision: AbsTopK explained variance no more than 0.02 below TopK; at least 1% of its selected activations negative. Both passed.
| Metric | TopK | AbsTopK |
|---|---|---|
| Explained variance | 0.966586 | 0.967736 |
| Negative activations | 0% | 49.2408% |
| Nonzero features per token | 100 | 100 |
Single-seed CPU comparison on tiny-stories-1M, with k=100, width 1,024 and 2.048 million training tokens per method. Both methods were trained inside one Validate run.
Run timing: The test was proposed 21 minutes 30 seconds after the request. Review included a notebook correction before the test was confirmed. Results posted 15 minutes 1 second after confirmation, or 56 minutes 19 seconds after the initial request, including the review interval.
This comparison ran on SAELens PR #5 and is separate from the earlier upstream contribution, which was reviewed and merged on its own.
The fork’s L0 metric counted positive activations and reported 50.88 for AbsTopK. Counting all nonzero activations gave 100. Negative activations explain the difference.
The merged upstream PR includes separate signed-metric fixes.
Does the result hold across more seeds, training budgets and concept-level evaluations?
9 targeted tests. Accuracy 0.475 vs 0.490 for LoRA in the reported setting.
262 custom-model tests passed. Mixed benchmark results disclosed.
12 reference checkpoints matched. 108 tests passed.
Regression tests and fixes for negative-active latents.
Maintainers reviewed and merged each contribution. Validate did not generate this historical evidence.
These are distinct from the four merged upstream contributions above. Fork-level PRs, coordination issues and in-review upstream work are not counted as accepted changes. This is a selected index of the examples documented in Outrider's README.
Outrider finds changes relevant to your codebase and drafts pull requests for review. Validate helps measure results against your criteria and carry that evidence into the next decision.
$ remyxai outrider init --repo owner/name --auto-interestFull setup ↗Validate: connect GitHub, then comment @remyx-ai validate on a PR. First completed run per App installation is free.
Benchmarks, review changes, and source links.
| CONTRIBUTION | PEFT · #3382 · MERGEDRiemannian-preconditioned LoRA |
PEFT · #3518 · MERGEDSuper-Tuning + Supra |
timm · #2767 · MERGEDEfficientViM |
SAELens · #736 · MERGEDAbsTopK bidirectional SAE |
|---|---|---|---|---|
| BAR | Integrate a safe, stable optimizer wrapper. |
Land a first-class PEFT tuner with reproducible checkpoints and native integration coverage. |
Reproduce every released checkpoint inside timm. |
Add a signed-activation SAE variant without breaking the library's existing metrics. |
| EVIDENCE | 9 tests0.475 vs 0.490 LoRA |
5 configsPEFT benchmark harness262 passedcustom-model tests Method and initialization suites also passed |
12 / 12checkpoints≤ 2.6e-6max |Δ|108tests |
6 commitsmethod, docs, testsregression testsnegative-active latents Per-token top-k by magnitude, sign preserved, asserted in tests |
| CHANGED | Rejected ambiguous optimizer closures. |
Reduced the first-release API and aligned checkpoint handling with PEFT conventions. |
Removed a non-portable dependency while preserving model behavior. |
Firing semantics in the synthetic evals moved to nonzero, and L1 now sums magnitudes, so bidirectional latents are counted correctly. |
| OUTCOME | Merged after 2 approvals Original author co-credited · below LoRA in this setting. |
Merged after multiple review rounds Paper author co-credited · benchmarks mixed. |
Merged in under 48 hours No public maintainer review thread. |
Merged after maintainer review 11 days from opening to merge · two review items fixed, two answered. |
Integrate a safe, stable optimizer wrapper.
9 tests · 0.475 versus 0.490 for LoRA.
Rejected ambiguous optimizer closures.
Merged after 2 approvals. Original author co-credited · below LoRA in this setting.
Land a first-class PEFT tuner with reproducible checkpoints and native integration coverage.
5 configurations in the PEFT benchmark harness · 262 custom-model tests passed.
Reduced the first-release API and aligned checkpoint handling with PEFT conventions.
Merged after multiple review rounds. Paper author co-credited · benchmarks mixed.
Reproduce every released checkpoint inside timm.
12 / 12 checkpoints · ≤ 2.6e-6 max |Δ| · 108 tests.
Removed a non-portable dependency while preserving model behavior.
Merged in under 48 hours. No public maintainer review thread.
Add a signed-activation SAE variant without breaking the library's existing metrics.
6 commits · method, docs and tests · regression tests on negative-active latents.
Synthetic-eval firing semantics moved to nonzero. L1 now sums magnitudes.
Merged after maintainer review. 11 days from opening to merge.
The contribution added a generic optimizer wrapper for Riemannian-preconditioned LoRA. Nine targeted tests covered the factory path, optimizer compatibility, DoRA compatibility, error handling, the paper formula, and bf16 finiteness.
The first implementation stopped after a pre-publication risk check. The narrower implementation proceeded. Matching the reference damping default moved GSM8K accuracy from 0.450 to 0.475, while default LoRA scored 0.490 in the same rank-32 MetaMathQA configuration. Review later exposed ambiguity around optimizer closures, so the final implementation rejects them.
The PR opened on Jul 1 and merged on Aug 3, 2026 after 33 days, 5 commits, and 2 approvals. The original author endorsed the revival and was co-credited. The disclosed benchmark did not beat LoRA in this setting. The result stayed visible when the contribution merged. Recorded Outrider execution across the stopped attempt and the rerun totals 33 minutes of machine time; that figure says nothing about total engineering effort, hours saved or the pace of maintainer review.
PR #3382 · authors' PR #1807 · coordination issue #3380 · final implementation commit · Riemannian LoRA paper
The contribution added Super-Tuning as a first-class PEFT method, with Supra as the hybrid low-rank mode. It included the tuner core, public registration, documentation, a worked example, method and initialization tests, generic integration coverage, and five benchmark configurations.
The five Super/Supra configurations ran against existing LoRA and Shira baselines in PEFT's MetaMathQA harness. Results were mixed: the best Supra row reached 0.4632 test accuracy versus 0.4784 for LoRA, with one run per configuration disclosed. A post-review custom-model run passed 262 tests with 10 skipped; the method and initialization suites also passed.
Review focused on integration and testing. The contributor and maintainer reduced the API surface and aligned checkpoint handling with PEFT conventions. The PR opened on Aug 5 and merged on Aug 21, 2026 after 10 commits and multiple review rounds. The maintainer ran the example before approval, and the paper's first author was co-credited. The evidence does not establish a general performance gain for Super-Tuning or Supra.
PR #3518 · coordination issue #3450 · author coordination · Super-Tuning paper
The contribution added EfficientViM M1 through M4 and their distilled variants as portable PyTorch models. It removed the reference implementation's non-portable helper dependency, preserved timm interfaces, and converted the released weights to safetensors.
All 12 released checkpoints loaded with zero missing and zero unexpected keys. Parameter counts matched the paper. Reference logits matched to a maximum absolute difference of 2.6e-6 across all variants. The targeted model suite passed 108 tests.
The PR opened on Sep 9 and merged on Sep 11, 2026. The public thread contains no maintainer review, so this page reports the evidence and timing without claiming why the merge was fast. The PR discloses AI assistance and human review of every line.
The contribution added AbsTopK as a registered SAE architecture in SAELens. Latents are selected per token by absolute value, with the sign preserved, so a feature can fire in either direction. The PR included the method, documentation and tests.
Review raised two correctness items that the change exposed in existing code: the synthetic evaluation calculators defined a firing latent as positive, and the L1 metric summed signed activations. Both were fixed in a follow-up commit with regression tests on negative-active latents. Two further items were answered with reference to the existing TopK path and the per-token selection asserted in tests.
The PR opened on Sep 21 and merged on Oct 2, 2026 after 6 commits and maintainer review. This page does not report file or test counts for this PR.
Merged means the linked upstream repository records the PR as merged. Checkpoint reproduced means the port loaded the released weights and matched the reference output within the stated numerical tolerance.
Remyx prepared these open-source contributions with AI assistance disclosed where required. Human contributors reviewed the work. Repository maintainers controlled review and merge. Validate did not produce the historical evidence on this page.