From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP

Adithya V. Ganesan, Vasudha Varadarajan, Oscar Kjell, Whitney R. Ringwald, Scott M. Feltman, Benjamin J. Luft, Roman Kotov, Ryan L. Boyd, H. Andrew Schwartz

July 2026 · Conference paper · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), 47160–47179

Read the paper →DOI

Abstract

While NLP typically treats documents as independent and unordered samples, in longitudinal studies, this assumption rarely holds: documents are nested within authors and ordered in time, forming person-indexed, time-ordered behavioral sequences. Here, we demonstrate the need for and propose a longitudinal modeling and evaluation paradigm that consequently updates four parts of the NLP pipeline: (1) evaluation splits aligned to generalization over people (cross-sectional) and/or time (prospective); (2) accuracy metrics separating between-person differences from within-person dynamics; (3) sequence inputs to incorporate history by default; and (4) model internals that support different coarseness of latent state over histories (pooled summaries, explicit dynamics, or interaction-based models). We demonstrate the issues ensued by traditional pipeline and our proposed improvements on a dataset of 17k daily diary transcripts paired with PTSD symptom severity from 238 participants, finding that traditional document-level evaluation can yield substantially different and sometimes reversed conclusions compared to our ecologically valid modeling and evaluation. We tie our results to a broader discussion motivating a shift from word-sequence evaluation toward behavior-sequence paradigms for NLP.

Figure 1. Flattened metrics can mask whether models learn people or dynamics.

Topics

Language-based assessment Assessment methods & guidelines Mental & physical health AI & machine learning

Keywords

Longitudinal NLP Behavioral sequences Evaluation Daily diaries PTSD

← All publications