Portrait of Julian Valline

Julian Valline

Doctoral researcher, LIST

I work on small open-weight language models for low-resource languages.

About

I am a doctoral researcher at the Luxembourg Institute of Science and Technology (LIST) and a PhD candidate at the University of Luxembourg, working in the ALT-EDIC LLMs4EU project. My research asks how far small, open-weight language models can go in languages with little training data, with Luxembourgish as the case study. In our LuxIT paper (LaTeLL 2026), we built a Luxembourgish instruction-tuning dataset of 227,507 pairs; fine-tuning on it improved 12 of 14 open models on Luxembourgish proficiency exams.

Before the PhD, I completed an M.Sc. in Natural Language Processing at Trier University and a B.Sc. in Artificial Intelligence at Johannes Kepler University Linz.

News

Earlier news
  • Started my PhD at the University of Luxembourg, based at LIST.

Publications

LuxIT: A Luxembourgish Instruction Tuning Dataset from Monolingual Seed Data

Julian Valline, Cedric Lothritz, Siwen Guo, Jordi Cabot

LaTeLL 2026 (Language Technologies for Low-resource Languages)

We generate Luxembourgish instruction-answer pairs from native Luxembourgish texts with DeepSeek-R1 and keep the 227,507 that pass an LLM-as-a-judge quality check. Fine-tuning 14 open models of up to 15B parameters on LuxIT raises their accuracy on Luxembourgish language exams by 5.37 percentage points on average (12 of 14 improve), and 9 of 14 also improve on five downstream NLP tasks.

BibTeX
@inproceedings{valline-etal-2026-luxit,
  title     = {{LuxIT}: A {Luxembourgish} Instruction Tuning Dataset from Monolingual Seed Data},
  author    = {Valline, Julian and Lothritz, Cedric and Guo, Siwen and Cabot, Jordi},
  booktitle = {Proceedings of the International Conference LAnguage TEchnologies for Low-resource Languages},
  month     = sep,
  year      = {2026},
  address   = {Fes, Morocco},
  pages     = {102--120},
  url       = {https://acl-bg.org/proceedings/2026/LaTeLL%202026/pdf/2026.latell-1.13.pdf}
}

Earlier work: Keli Du, Sarah Ackerschewski, Uygar Navruz, Nazan Sınır, Julian Valline, Christof Schöch. Reconstructing Shuffled Text. Bad Results for NLP, but Good News for Using In-Copyright Texts. Journal of Computational Literary Studies 4(1), 2025.

Open source and datasets

  • Dataset

    LuxIT, Wikipedia part

    46,072 synthetic Luxembourgish instruction-answer pairs generated from Luxembourgish Wikipedia articles.

    Hugging Face · CC BY-SA 4.0

  • Code

    LuxIT

    The pipeline behind LuxIT: data generation with DeepSeek-R1, LLM-as-a-judge filtering, LoRA fine-tuning across eight model families, and evaluation on Luxembourgish exams and NLP tasks.

    GitHub · MIT licence

Experience

  1. – present

    Doctoral researcher

    Luxembourg Institute of Science and Technology (LIST) · Esch-sur-Alzette, Luxembourg

    • PhD candidate at the University of Luxembourg, in the ALT-EDIC LLMs4EU project.
    • Built LuxIT, 227,507 synthetic Luxembourgish instruction-answer pairs, and used it to fine-tune and evaluate 14 open models.
    • Released the pipeline code (MIT licence) and a 46,072-pair part of the dataset (CC BY-SA 4.0).

Education

  1. – present

    PhD candidate

    University of Luxembourg · Esch-sur-Alzette, Luxembourg

    • Research at LIST in the ALT-EDIC LLMs4EU project.
    • Supervisors: Jordi Cabot and Cedric Lothritz
  2. Summer schools

    • Lisbon Machine Learning School (LxMLS 2025)
    • International Artificial Intelligence Summer School (IAISS 2025)
  3. –

    M.Sc. Natural Language Processing

    Trier University · Trier, Germany

    • Thesis: "Synthesizing an Instruction Tuning Dataset for Luxembourgish"
  4. –

    B.Sc. Artificial Intelligence

    Johannes Kepler University Linz · Linz, Austria

    • Thesis: "Zero-shot transfer of monolingual language models"