The paper establishes a mathematical correspondence between decision trees and diffusion models, revealing a shared optimization principle called GTSM. The proposed TreeFlow achieves 2x speedup in tabular data generation, while DSMTree distills hierarchical logic into neural networks within 2% of teacher performance.
The paper presents a theoretical framework unifying decision trees and diffusion models. It proves a mathematical correspondence between the two in limiting regimes and derives a shared optimization principle called Global Trajectory Score Matching (GTSM).
Decision trees are discrete and hierarchical, while diffusion models are continuous and dynamic. These seemingly disparate model classes are shown to have deep connections, expanding the unified understanding of machine learning.
Beyond theoretical unification, the work introduces practical instantiations (TreeFlow and DSMTree) that demonstrate improved performance in tabular data generation and model distillation. This suggests potential synergies between tree-based methods (e.g., gradient boosting) and diffusion models.
The paper's mathematical unification of decision trees and diffusion models was acknowledged, but some comments questioned its mathematical rigor and practical usefulness. On the other hand, there were opinions highly evaluating practical achievements such as the generation quality and speed improvement of TreeFlow applying diffusion models to tabular data, and the performance proximity of the distillation technique DSMTree. Practical concerns like code availability were also raised.