Procurement fraud generates significant economic and social losses worldwide. Despite the efforts of auditing authorities, manual inspection proves inefficient. Due to the high volume of processes and the time required for each analysis, detection rates are low. Computer models have emerged as a promising way to automate fraud detection and prediction. This thesis introduces a novel framework for automatically extracting red flags in public procurement using deep learning and natural language processing (NLP) techniques. The study addresses three main gaps: the lack of automated solutions for red flag detection, the absence of labeled textual databases, and the scarcity of domain-specific language models. The research presents a framework comprising several models for tasks such as section classification, object classification, named entity recognition, and red flag detection. To address the lack of annotated data, the study compiled multiple datasets with thousands of documents related to public procurement, including more than 420,000 documents in Portuguese. A key contribution is the development of HelBERT, a pre-trained language model for the public procurement domain, trained on a large corpus of Portuguese documents. The findings show that HelBERT and its variants outperform general-purpose and legal-domain models in downstream tasks, such as red flag classification and semantic similarity. For example, HelBERT achieved an F1 score of 94.91% in red flag classification and demonstrated the highest accuracy (97.04%) in object classification using keyphrases.