Elamur Research⌕
☰
home / papers / 2026 / Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models

Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models

Paper · 2026 · Zefang Liu, Nam H Nguyen, Yinzhu Quan et al.

Summary

LLMs process discrete tokens, but events in real sequences happen in continuous time. How to turn timestamps or inter-event gaps into tokens ("temporal tokenization") has received little systematic study. This paper compares five families of encodings: plain numeric strings, float32 byte tokens (as in Language-TPP), Gregorian calendar tokens, uniform scale binning in linear or log space, and a new residual scalar quantization (RSQ) scheme. RSQ stacks K-Means codebooks over successive residuals.

Each variant is fine-tuned on Llama-3.2-1B with QLoRA. The data are five public TPP-LLM benchmarks whose interval distributions range from smooth log-normal to multi-modal to sharply spiky. Next-event-type accuracy barely changes across tokenizers, but time-prediction error depends strongly on how well the encoding fits the data's distribution.

The study offers practical guidance on choosing a time encoding. It also shows that plain next-token fine-tuning, with no TPP heads or likelihood integrals, is a simpler and competitive alternative to hybrid point-process models.

Key points

  • Data: five TPP-LLM datasets (hosted on Hugging Face under tppllm), covering Amazon Review, US Earthquake, NYC Taxi, Chicago Crime and Stack Overflow. They range from about 30K to about 362K events.
  • Configurations and cost:
    • Numeric strings use 6 decimals and take roughly 4 tokens.
    • Byte encoding always uses 4 tokens.
    • Calendar tokens come in day and second resolution, absolute or relative, using 3 or 6 tokens.
    • Scale binning uses 256 bins and a single token.
    • RSQ uses either 1 level of 256 bins or 4 levels of 64 bins.
  • Evaluation: metrics are next-type accuracy, interval RMSE and tokens per time value. Results are averaged over 5 seeds.
  • Best time-prediction strategy by dataset:
    • Log-scale strategies win on Stack Overflow, Chicago Crime and Amazon Review: Scale Bin (Log) on Chicago Crime, and RSQ (Log, L4) on the other two.
    • Absolute Calendar (Second) wins on NYC Taxi, since it captures daily and hourly cycles.
    • Byte wins on US Earthquake.
  • Efficiency: single-token log binning and RSQ offer the best balance between accuracy and token cost. Code is available at github.com/CapitalOne-Research/temporal-tokenization.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgia Institute of TechnologyAtlanta · United States133 works · 24,497 citations🏢Capital OneTysons Corner · United States6 works · 16 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Tokenization & segmentation · 119 works

Cite

Cite as Zefang Liu, Nam H Nguyen, Yinzhu Quan and Shi-Xiong Zhang. 2026. Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models. Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.gem-main.5

@inproceedings{liu2026temporal,
  title = {{Temporal Tokenization Strategies for Event Sequence Modeling with Large Language Models}},
  author = {Zefang Liu, Nam H Nguyen, Yinzhu Quan and Shi-Xiong Zhang},
  booktitle = {Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)},
  year = {2026},
  doi = {10.18653/v1/2026.gem-main.5},
  url = {https://doi.org/10.18653/v1/2026.gem-main.5}
}

References

2 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gagan Bhatia, Maxime Peyrard, and Wei Zhao. 2025. Date fragments: A hidden bottleneck of tokenization for temporal reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3201–3219, Suzhou, China. Association for Computational Linguistics. find
  2. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 10088–10115. \#\# Limitations

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-temporal-tokenization-strategies-event-sequence/