Big GCVAE: decision-making with adaptive transformer model for failure root cause analysis in semiconductor industry - Mines Saint-Étienne Access content directly
Journal Articles Journal of Intelligent Manufacturing Year : 2024

Big GCVAE: decision-making with adaptive transformer model for failure root cause analysis in semiconductor industry

Abstract

Pre-trained large language models (LLMs) have gained significant attention in the field of natural language processing (NLP), especially for the task of text summarization, generation, and question answering. The success of LMs can be attributed to the attention mechanism introduced in Transformer models, which have outperformed traditional recurrent neural network models (e.g., LSTM) in modeling sequential data. In this paper, we leverage pre-trained causal language models for the downstream task of failure analysis triplet generation (FATG), which involves generating a sequence of failure analysis decision steps for identifying failure root causes in the semiconductor industry. In particular, we conduct extensive comparative analysis of various transformer models for the FATG task and find that the BERT-GPT-2 Transformer (Big GCVAE), fine-tuned on a proposed Generalized-Controllable Variational AutoEncoder loss (GCVAE), exhibits superior performance in generating informative latent space by promoting disentanglement of latent factors. Specifically, we observe that fine-tuning the Transformer style BERT-GPT2 on the GCVAE loss yields optimal representation by reducing the trade-off between reconstruction loss and KL-divergence, promoting meaningful, diverse and coherent FATs similar to expert expectations.
Embargoed file
Embargoed file
0 9 14
Year Month Jours
Avant la publication
Wednesday, April 2, 2025
Embargoed file
Wednesday, April 2, 2025
Please log in to request access to the document

Dates and versions

emse-04530213 , version 1 (04-04-2024)
emse-04530213 , version 2 (22-04-2024)

Identifiers

Cite

Kenneth Ezukwoke, Anis Hoayek, Mireille Batton-Hubert, Xavier Boucher, Pascal Gounet, et al.. Big GCVAE: decision-making with adaptive transformer model for failure root cause analysis in semiconductor industry. Journal of Intelligent Manufacturing, 2024, ⟨10.1007/s10845-024-02346-x⟩. ⟨emse-04530213v2⟩
40 View
11 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More