results highlight the importance of previously overlooked design choices, and raise questions about the source RoBERTa has almost similar architecture as compare to BERT, but in order to improve the results on BERT architecture, the authors made some simple design changes in its architecture and