Electrocardiogram (ECG), a non-invasive diagnostic technique, used for detecting cardiac arrhythmia. From last decade industry dealing with biomedical instrumentation and research, demanding an advancement in its ability to distinguish different cardiac arrhythmia. Atrial Fibrillation (AF) is an irregular rhythm of the human heart. During AF, the atrial moments are quicker than the normal rate. As blood is not completely ejected out of atria, chances for the formation of blood clots in atrium. These abnormalities in the heart can be identified by the changes in the morphology of the ECG. The first step in the detection of AF is preprocessing of ECG, which removes noise using filters. Feature extraction is the next key process in this research. Recent feature extraction methods, such as Auto Regressive (AR) modeling, Magnitude Squared Coherence (MSC) and Wavelet Coherence (WTC) using standard database (MIT-BIH), yielded a lot of features. Many of these features might be insignificant containing some redundant and non-discriminatory features that introduce computational burden and loss of performance. This paper presents fast Conjugate Symmetric Sequency Ordered Complex Hadamard Transform (CS-SCHT) for extracting relevant features from the ECG signal. The sparse matrix factorization method is used for developing fast and efficient CS-SCHT algorithm and its computational performance is examined and compared to that of the HT and NCHT. The applications of the CS-SCHT in the ECG-based AF detection is also discussed. These fast CS-SCHT features are optimized using Hybrid Firefly and Particle Swarm Optimization (FFPSO) to increase the performance of the classifier.