The Tree Based Linear Regression Model for Hierarchical Categorical Variables

Emilio Carrizosa, Laust Hvas Mortensen, Dolores Romero Morales, M. Remedios Sillero-Denamiel*

*Corresponding author for this work

Research output: Contribution to journalJournal articleResearchpeer-review

1 Downloads (Pure)

Abstract

Many real-life applications consider nominal categorical predictor variables that have a hierarchical structure, e.g. economic activity data in Official Statistics. In this paper, we focus on linear regression models built in the presence of this type of nominal categorical predictor variables, and study the consolidation of their categories to have a better tradeoff between interpretability and fit of the model to the data. We propose the so-called Tree based Linear Regression (TLR) model that optimizes both the accuracy of the reduced linear regression model and its complexity, measured as a cost function of the level of granularity of the representation of the hierarchical categorical variables. We show that finding non-dominated outcomes for this problem boils down to solving Mixed Integer Convex Quadratic Problems with Linear Constraints, and small to medium size instances can be tackled using off-the-shelf solvers. We illustrate our approach in two real-world datasets, as well as a synthetic one, where our methodology finds a much less complex model with a very mild worsening of the accuracy.
Original languageEnglish
Article number117423
JournalExpert Systems with Applications
Volume203
Number of pages36
ISSN0957-4174
DOIs
Publication statusPublished - Oct 2022

Bibliographical note

Published online: 4 May 2022.

Keywords

  • Hierarchical categorical variables
  • Linear regression models
  • Accuracy vs. model complexity
  • Mixed integer convex quadratic problem with linear constraints

Cite this