Ledom: An Open and Fundamental Reverse Language Model

The Reverse Language Model (RLM) employs a reverse-temporal autoregressive pre-training objective. An input text sequence, $\bm{x}=(x_{1},x_{2},$ $\dots,x_{T})$, tokenized using standard FLM tokenizers for compatibility, is reversed to $\bm{x}^{\mathrm{reverse}}=(x_{T},$ $x_{T-1},\dots,x_{1})$. The RLM models the joint probability of the original sequence in reverse as:

where $\theta_{\mathrm{RLM}}$ demotes the model parameters. Such prediction contrasts with a conventional FLM:

The reversed conditioning in RLM leads to distinct characteristics as follows:

• •

  Reverse Contextualization: Hidden states at position $t$, $\mathbf{h}_{t}=f(x_{t+1},\dots,x_{T};\theta)$, depend solely on tokens that appeared later in the sequence.

• •

  Initial Prediction Ambiguity: The first token predicted by the RLM (corresponding to $x_{T}$ in the original sequence) is unconditional, as no future tokens are available: $\mathbb{P}(x_{T}\mid\varnothing;\theta_{\mathrm{RLM}})$.

• •

  Backward Gradient Flow: Gradients propagate from the end to the start of the sequence, influencing training dynamics differently from FLMs.

Using standard tokenization ensures token boundary consistency with FLMs, facilitating direct token-level distribution comparisons.

Our pre-training corpus, $\mathcal{D}$, totals 435B tokens, comprising three components: (1) $\mathcal{D}_{\text{General}}$: 284B tokens from DCLM (Li et al., 2024), a deduplicated and domain-balanced general text dataset. (2) $\mathcal{D}_{\text{Math}}$: 102B tokens to enhance numerical and formal logic reasoning. (3) $\mathcal{D}_{\text{Code}}$: 48B tokens from MAP-Neo (Zhang et al., 2024) for improved structural reasoning. Detailed token statistics for each category are provided in Appendix A.1.

Both the RLM (Ledom) and the comparative FLM utilize an identical Transformer decoder architecture (Vaswani et al., 2023), instantiated at 2B and 7B parameter scales. Key architectural enhancements include Multi-Query Attention, Rotary Positional Embeddings (RoPE) (Su et al., 2023), RMSNorm normalization (Zhang and Sennrich, 2019), and SwiGLU activation functions (Shazeer, 2020). Specific model architectural details (e.g., layers, heads) are available in Table 1.

For training, we employed the AdamW optimizer and a cosine learning rate schedule with a warmup period of 0.55% of total training steps. Full hyperparameter details and the learning rate schedule equation are in Appendix 2.3.

As shown in Figure 4, the RLM exhibits slower convergence and a higher asymptotic training loss compared to the FLM. We attribute this to increased predictive uncertainty inherent in reverse-temporal modeling. The RLM must infer initial context implicitly, as defined by its loss function:

To assess Ledom as a general-purpose foundation model and compare it with conventional FLMs, we adopted a standardized few-shot evaluation approach, following Brown et al. (2020). A key modification was made to align with how Ledom is typically pre-trained: we universally reversed the token sequences for all parts of each task instance, including the query, any intermediate reasoning steps, and the final answer.

Formally, if a standard task instance has a question $Q=\{q_{1},\dots,q_{n}\}$, optional reasoning steps $S=\{s_{1},\dots,s_{m}\}$, and an answer $A=\{a_{1},\dots,a_{k}\}$, our method uses their reversed counterparts: $Q^{R}=\{q_{n},\dots,q_{1}\}$, $S^{R}=\{s_{m},\dots,s_{1}\}$, and $A^{R}=\{a_{k},\dots,a_{1}\}$. The few-shot prompt given to Ledom consists of $N$ demonstration instances followed by the token-reversed test question $Q_{\text{test}}^{R}$. Each demonstration $D_{i}$ is formatted as:

These demonstrations are concatenated, and the prompt ends with $Q_{\text{test}}^{R}\texttt{ :Question}$. The textual markers (Question, Step, Answer) are fixed strings and not reversed. Ledom is then tasked with generating the token-reversed steps $S_{\text{test}}^{R}$ (if applicable) and answer $A_{\text{test}}^{R}$. Further details on specific prompts are in Figure 5.

The main reason for this comprehensive token reversal is to present tasks to Ledom in a format that closely matches its pre-training on reverse sequences, potentially allowing it to better use its learned capabilities. We acknowledge that evaluating on tasks requiring inherently forward reasoning might not be ideal for models designed for reverse sequence prediction. However, our goal is not to claim Ledom’s superiority for all such tasks, but to explore its underlying abilities and adaptability as a general foundation model when inputs are appropriately transformed.

We evaluated Ledom on eight diverse benchmarks from the OpenCompass suite (Contributors, 2023), covering: general reasoning and commonsense, code generation, world knowledge and question answering, and mathematical reasoning. A detailed description of each benchmark is provided in Appendix B. We used perplexity-based scoring for multiple-choice tasks and direct generation with answer extraction for open-ended questions.

Overall, the approach of training models to predict sequences in reverse order shows promise. As shown in Table 2, our Ledoms achieve results comparable to FLMs (trained on the same data) on several benchmarks, indicating their potential as foundational models. Our key observations are:

A key feature of Ledom is its ability to construct coherent prior sequences that logically lead to a known outcome. In abductive reasoning, for instance, Ledom generates plausible causal chains for a given final state, effectively reconstructing "what must have happened." Similarly, in story generation, the model shows strong narrative skill when creating a lead-in to a known conclusion, such as a fairytale ending. These abilities make reverse generation useful for applications like building alternative stories, simulation-based inference, and generating explanations.

In mathematical reasoning, Ledom often works backward from a final result to figure out intermediate steps and needed operations. This process is similar to how humans sometimes reason when reverse-engineering a problem or finding the possible source of an error. Such abilities suggest strong potential for Ledom in educational tools, debugging help, and other situations where breaking down a result into its preceding logical steps is important. The use of Ledom in math reasoning tasks is explored further in Section 6.

Even though it processes text in reverse, Ledom maintains clear semantic understanding and performs reliably on classification and inference tasks. In few-shot sentiment classification, it successfully generalizes from example patterns, showing it can grasp task structure even with unconventional generation formats. Likewise, in definitional reasoning and commonsense queries, Ledom maintains accuracy and consistency, suggesting that core semantic understanding is largely unaffected by the processing direction. An especially promising scenario is the question generation task. Given a known answer and the corresponding supporting reasons, Ledom can produce natural, well-formed questions. It is helpful for automatically creating QA datasets and educational content, where starting from answers or known concepts is often more practical than designing questions manually.

The case studies also show potential safety risks specific to reverse generation. As one example illustrates (Table 3, Unsafe Prompt), Ledom produced dangerous instructional content from a prompt that would usually trigger safeguards in FLMs. This suggests that existing safety filters, mostly designed for left-to-right generation, may be insufficient for reverse decoding. Ensuring safety in reverse models requires dedicated measures, possibly including bidirectional alignment or adversarial training.

Furthermore, Ledom shows potential to address certain generalization problems common in FLMs, especially the "reversal curse," where models fail to infer inverse relations (e.g., "B is A" from "A is B"). Reverse models seem better at completing such inverse statements, suggesting a potential for forward and reverse processing to complement each other. This finding opens the possibility for hybrid systems that combine both forward and reverse decoding to achieve more balanced and symmetric language understanding.

In summary, reverse language models like Ledom offer more than just a change in direction; they provide different ways for reasoning, generation, and question construction. While they present unique safety issues, they also offer valuable behaviors that FLMs often lack and can complement them. Understanding these characteristics well can help design future model architectures and training methods that effectively combine the strengths of both forward and reverse modeling.

Ledom introduces a unique paradigm by explicitly modeling tokens conditioned on subsequent (future) context. To leverage Ledom’s distinctive posterior reasoning capabilities, we propose a novel inference mechanism called Reverse Reward. This approach harnesses backward-conditioned probability estimates from Ledom as posterior evaluation signals, enabling refined output generation from conventional FLMs.

Formally, given an input prompt $\bm{x}=(x_{1},x_{2},\dots,x_{T})$ and an FLM-generated candidate response $\bm{y}=(y_{1},y_{2},\dots,y_{M})$, the reverse reward is defined as the likelihood of the input sequence $\bm{x}$ conditioned on the generated response $\bm{y}$:

where the product of probabilities reflects the Ledom’s capability to assess token likelihood based on future context within the prompt $\bm{x}$ and the full candidate response $\bm{y}$.

To balance backward contextualization from Ledom with forward predictive power from the FLM, we define a unified bidirectional reward framework:

where $\lambda\in[0,1]$ controls the relative contribution of the reverse reward. We propose two inference strategies utilizing this combined reward in the next section. Figure 2 illustrates the Reverse Reward mechanism with a query requiring two steps (thoughts) to reach the answer.

More details of the experimental settings for mathematical reasoning can be found in D.2.

Table 4 summarizes our comprehensive evaluation results. For Beam Search strategy, considering the computational cost, we only use OpenMath2 to conduct experiments to demonstrate that Reverse Reward is effective under different granularities. Key findings include:

We investigate the impact of candidate response sampling size ($N$) in the Step-Level Decoding via Beam search strategy. Specifically, we conduct experiments with FLM as base model and various $N$ from 1 to 64 on MATH-500 and GSM8K requiring multi-step math reasoning. Results shown in Figure 3 demonstrates that higher $N$ exhibits better performance across both datasets, suggesting that our Reverse Reward has the potential to boost the performance of the base model with the backward signal provided by Ledom at each reasoning step. It also shows a trade-off between computational cost and performance gains.

We conduct an illustrative case study to demonstrate concrete reasoning improvements provided by reranking and beam search decoding with Reverse Reward (Appendix D.4). Specifically, the examples highlight that the Reverse Reward can capture details missed by FLM or verify the generation of FLM from the backward-contextualized evaluation of Ledom, significantly improving the correctness, coherence, and reasoning depth in FLM-generated mathematical solutions.

Our experimental results empirically validate the theoretical motivation behind the Reverse Reward framework. The integration of forward and reverse language modeling distinctly improves generative performance, particularly in tasks involving sophisticated reasoning and logical inference. These findings open promising avenues for future exploration of reverse autoregressive modeling as a complementary tool for enhancing traditional forward-based generation methodologies.

In conclusion, the proposed Reverse Reward paradigm robustly establishes the practical utility of Reverse Language Models in NLP, significantly advancing both theoretical understanding and practical methodologies for improved reasoning-oriented text generation.