Abstract
Detecting fraudulent patterns and irregular operational anomalies in high-dimensional financial transaction data represents a persistent challenge due to extreme class imbalance, multi-modal feature correlations, and dynamic adversarial shifts. Classical deep generative models, such as Variational Autoencoders (VAEs), often suffer from posterior collapse and struggle to capture intricate, non-linear dependencies across hundreds of continuous and categorical transaction features. In this work, we propose a novel Quantum-Inspired Variational Autoencoder (QI-VAE) tailored for unsupervised anomaly detection in complex financial streams. Our architecture integrates classical tensor network representations—specifically Matrix Product States (MPS)—into the latent generative pipeline, coupled with a density matrix formalism that enforces quantum-inspired unitary rotations and von Neumann entropy regularization across the latent manifolds. By simulating quantum superposition and entanglement principles on classical computing hardware, the QI-VAE effectively maps high-dimensional transactions into a compact, highly expressive latent state space where normal behaviors form constrained, coherent trajectories. Evaluated on large-scale real-world financial transaction benchmarks, including the Credit Card Fraud Detection and IEEE-CIS Fraud datasets, the proposed QI-VAE framework achieves an average improvement of 5.8% in Area Under the Precision-Recall Curve (PR-AUC) and a 4.2% gain in F1-score over state-of-the-art classical generative and tree-based baselines, while requiring up to 35% fewer trainable parameters. These findings demonstrate the viability of quantum-inspired inductive biases for enterprise financial monitoring systems.