One of the significant problems in cosmology is the morphological classification of galaxies in order to gain insights about the origin and evolution of our Cosmos. Here, we investigate the classification accuracy of several modern convolutional neural network architectures (some of them pretrained on non-astronomical data and others trained from scratch to galaxy data), and of transformers with primary goal to understand their performance on conditions of fast training (i.e., in a few rounds). We experimented with Transfer Learning, and Vision Transformers (ViT). We contrasted their performance with that of traditional Convolutional Neural Networks (CNN) methods, i.e., AlexNet, and VGG. Our results show a 75% classification accuracy (by the ViT and VGG), beating the rest of the competitors. Moreover, one pretrained VGG model showed also excellent performance.