Getting Started

A quick-start guide for fine-tuning LLMs with EulerForge. For detailed explanations of each injection strategy, refer to the strategy-specific tutorials.

1. Installation and Environment

Prerequisites

• Python >= 3.9
• PyTorch >= 2.1
• CUDA-capable GPU (recommended; CPU also works for testing)

Installation

# Clone the repository
git clone <repo_url>
cd eulerforge

# Install in editable mode
pip install -e .

# For development (includes tests)
pip install -e ".[dev]" pytest

Verify Installation

python -c "import eulerforge; print('OK')"
eulerforge train --help

2. Data Preparation

First, run the Data Preprocessing Guide. It converts the raw data in data/ into EulerForge standard raw JSONL format.

Provided Data and Purposes

Using Raw Data

Specify data.format=raw and the data will be automatically tokenized during training:

eulerforge train --preset configs/presets/<preset>.yml \
    --set data.format=raw \
    --set data.task=sft \
    --set data.path=data/sft_10k_raw.jsonl \
    --set data.max_length=512

Details: Data Preprocessing Guide

3. Core Concepts: Where / What / When

EulerForge organizes fine-tuning from three perspectives:

Each strategy tutorial explains these three perspectives step by step.

4. Backbone Adapters

The correct adapter is automatically selected based on the backbone configuration key:

5. Choosing an Injection Strategy

Strategy Comparison Table

Strategy Tutorials

• Plain LoRA Tutorial -- The most basic LoRA fine-tuning
• LoRA MoE Tutorial -- Router + multiple LoRA experts
• FFN MoE Expert LoRA Tutorial -- Convert dense models to MoE
• Native MoE Expert LoRA Tutorial -- Mixtral-specific fine-tuning

Training Method Tutorials

• DPO Training Guide -- Preference-based alignment training (reference model)
• ORPO Training Guide -- Preference-based alignment (no reference model needed)
• RM Training Guide -- Reward model training (Bradley-Terry)
• PPO/RLHF Guide -- PPO-based RLHF pipeline

Model-Specific Tutorials

• LLaMA Fine-Tuning Guide -- Llama-3.2-1B SFT + TinyLlama DPO

Training Method Selection Table

Which Strategy Should You Choose?

Is the model already an MoE architecture? (e.g., Mixtral)
  ├── Yes → native_moe_expert_lora
  └── No (Dense model)
        ├── Simple fine-tuning? → dense_lora
        ├── Multi-task/adaptive? → mixture_lora
        └── Convert to MoE structure? → moe_expert_lora

6. Phase Schedule Overview

The phase schedule controls when to train which parameter groups. Define it declaratively in training.phases.

Parameter Groups

Phase Patterns by Strategy

For detailed phase configuration and timelines, refer to the "When" section of each strategy tutorial.

7. CLI Quick Start

Basic Training (raw data)

eulerforge train --preset configs/presets/<preset>.yml \
    --set data.format=raw \
    --set data.task=sft \
    --set data.path=data/sft_10k_raw.jsonl \
    --set data.max_length=512

Available Presets

Configuration Overrides

Override any configuration value using dot-path notation:

eulerforge train --preset configs/presets/qwen3.5_0.8b_dense_lora_sft.yml \
    --set data.format=raw \
    --set data.task=sft \
    --set data.path=data/sft_10k_raw.jsonl \
    --set data.max_length=512 \
    --set training.lr=2e-5 \
    --set injection.lora_r=32

Useful CLI Options

For a full CLI reference, see the CLI Documentation.

8. Running Benchmarks

eulerforge bench --target-dir /path/to/checkpoint \
    --ref-model Qwen/Qwen3.5-0.8B-Base \
    --test-data /path/to/test.jsonl \
    --output-file results.jsonl

9. Common Troubleshooting

For strategy-specific troubleshooting, refer to the "Debugging and Troubleshooting" section of each tutorial.

10. Scratch Pretraining (pretrain)

To train a newly assembled model from scratch (e.g., built with EulerStack) rather than an existing HF model, use the eulerforge pretrain command.

eulerforge pretrain --preset configs/presets/pretrain/eulerstack_hybrid_moe.yml

pretrain is a completely separate pipeline from train, performing full-parameter causal LM training. It does not use LoRA injection or phase scheduling.

Details: 17_pretrain.md

11. Training Pipeline -- From SFT to PPO

EulerForge supports 5 training types. Always start with SFT first:

SFT → DPO (or ORPO) → Deploy       # Most common (2 stages)
SFT → RM → PPO → Deploy             # Full RLHF (3 stages)
SFT → ORPO → RM → PPO → Deploy      # ORPO-based full RLHF

Warning: Applying DPO/ORPO/RM/PPO directly to a base model will degrade performance. Always provide instruction-following ability via SFT first, then proceed with preference learning.

The checkpoint from each stage becomes the model input for the next stage.

Details: 18_training_pipeline.md