The massive consumption of social media applications by the huge number of online users producing the feedback of services, products, situations, and events leads to the evaluation of the sentiment extraction tasks. At present, several deep learning architectures such as Long short-term memory(LSTM), convolutional neural network(CNN) are preferred prevalently in sentiment classification problems. In this present study, an ensemble comprising of a transformer-based deep learning approach is proposed for sentiment extraction tasks. To evaluate the model, a dataset from Kaggle is considered for training and compared with the LSTM with attention and glove embedding and CNN-glove models. The evaluation of the models is analyzed with the performance metrics such as accuracy, precision, recall, F1-score, and error rate. By comparing the performance of the proposed model with that of past studies, the proposed model offers better sentiment extraction performance.