Comparitive Analysis of Gradient Boosting and Transformer Based Models for Binary Classification in Tabular Data

This study compares the classification performance of the Gradient Boosting (XGBoost), and Transformer based model with multi-head self-attention for Tabular Data. While the methods exhibit broadly similar performance, the Transformer model particularly excels in Recall by about 8% showing that it would be better suited to applications such as Fraud Detection in Payment processing and Medical Diagnostics.