/ approach
Studying how models improve.
AlgoQX’s research direction centers on the feedback loop behind language-model behavior. We are interested in human preferences, verifiable outcomes and independent evaluation, together with the systems needed to connect them.
/01 RLHFReinforcement learning from human feedback
Human judgment where it matters
Reinforcement learning from human feedback
Many useful qualities are difficult to reduce to an executable check. We are interested in preference collection, clear rubrics, disagreement and the choices involved in using feedback for post-training.
- preference collection
- rubrics
- disagreement
- post-training choices
/02 RLVRReinforcement learning with verifiable rewards
Verification where it is meaningful
Reinforcement learning with verifiable rewards
Coding, SQL and bounded tool-use tasks can produce outcomes we can inspect through execution. The challenge is to design checks that reflect the intended task and remain useful beyond the easiest examples.
explanatory A verifier is a component to question, not an oracle.
/03 evaluation
Evaluation beyond reward
A training signal and a final capability measure answer different questions. We want to examine held-out behavior, failure cases and counterexamples alongside training results.
- training reward asks
- Did the output score well on the signal being optimized?
- independent evaluation asks
- Does the behavior hold on held-out tasks, failure cases and counterexamples?
/04 systems
Practical systems for experiments
Efficient inference, reliable execution and clear resource boundaries can make experimentation more accessible. We see these as parts of the research process, with their own assumptions and limitations.
explanatory Rollouts run side by side, each inside explicit resource bounds.
These are research directions. Public notes and releases will identify the methods, evidence and status of specific work.
/ glossary
Terms, in plain words.
- RLHFReinforcement learning from human feedback
- Training a model with a reward signal derived from human preferences, often through a learned reward model.
- RLVRReinforcement learning with verifiable rewards
- Training where the reward comes from a check that can be executed, such as running tests or comparing a query result.
- Reward modelA learned stand-in for human preference
- A model trained to predict which output people would prefer, used as the reward during training.
- DPODirect preference optimization
- A post-training method that learns from preference pairs directly, without a separate reward model or a reinforcement-learning loop.
- VerifierThe check that decides correct
- The procedure that decides whether an output counts as solved. It can be incomplete, wrong or exploitable, so it deserves its own scrutiny.
- RolloutOne attempt at a task
- A single run of the model on a task, from prompt to final output, during training or evaluation.
- Held-out setTasks kept out of training
- Examples the model never trained on, used to check whether an improvement transfers beyond the training signal.
/ contact
Have a question worth testing?
If you are working on post-training, verification or evaluation, tell us about the task and the constraints around it.
Please share a short description first. Avoid sending confidential data in an initial enquiry.