Comparing SMILES and SELFIES tokenization for enhanced chemical language modeling

Leon, Miguelangel (avtor), Perezhohin, Yuriy (avtor), Peres, Fernando (avtor), Popovič, Aleš (avtor), Castelli, Mauro (avtor)

Life sciences research and experimentation are resource-intensive, requiring extensive trials and considerable time. Often, experiments do not achieve their intended objectives, but progress is made through trial and error, eventually leading to breakthroughs. Machine learning is transforming this traditional approach, providing methods to expedite processes and accelerate discoveries. Deep Learning is becoming increasingly prominent in chemistry, with Convolutional Graph Networks (CGN) being a key focus, though other approaches also show significant potential. This research explores the application of Natural Language Processing (NLP) to evaluate the effectiveness of chemical language representations, specifically SMILES and SELFIES, using tokenization methods such as Byte Pair Encoding (BPE) and a novel approach developed in this study, Atom Pair Encoding (APE), in BERT-based models. The primary objective is to assess how these tokenization techniques influence the performance of chemical language models in biophysics and physiology classification tasks. The findings reveal that APE, particularly when used with SMILES representations, significantly outperforms BPE by preserving the integrity and contextual relationships among chemical elements, thereby enhancing classification accuracy. Performance was evaluated in downstream classification tasks using three distinct datasets for HIV, toxicology, and blood–brain barrier penetration, with ROC-AUC serving as the evaluation metric. This study highlights the critical role of tokenization in processing chemical language and suggests that refining these techniques could lead to significant advancements in drug discovery and material science.

• Jezik: Angleški jezik
• Ključne besede: tokenization, language modelling, natural language processing
• Vrsta gradiva: Članek v reviji
• Tipologija: 1.01 - Izvirni znanstveni članek
• Organizacija: FRI - Fakulteta za računalništvo in informatiko
• Status publikacije: Objavljeno
• Različica publikacije: Objavljena publikacija
• Št. strani: 13 str.
• Številčenje: Vol. 14, art. 25016
• PID: 20.500.12556/RUL-183997
• UDK: 004.85:81'322
• ISSN pri članku: 2045-2322
• DOI: 10.1038/s41598-024-76440-8
• COBISS.SI-ID: 212901379
• Datum objave v RUL: 24.06.2026

Gradivo je del revije

• Naslov: Scientific reports
• Skrajšan naslov: Sci. rep.
• Založnik: Springer Nature
• ISSN: 2045-2322
• COBISS.SI-ID: 18727432

Licence

• Licenca: CC BY-NC-ND 4.0, Creative Commons Priznanje avtorstva-Nekomercialno-Brez predelav 4.0 Mednarodna
• Povezava: http://creativecommons.org/licenses/by-nc-nd/4.0/deed.sl
• Opis: Najbolj omejujoča licenca Creative Commons. Uporabniki lahko prenesejo in delijo delo v nekomercialne namene in ga ne smejo uporabiti za nobene druge namene.

Sekundarni jezik

• Jezik: Slovenski jezik
• Ključne besede: tokenizacija, jezikovno modeliranje, obdelava naravnega jezika

Projekti

• Financer: FCT - Fundação para a Ciência e a Tecnologia, I.P.
• Številka projekta: UIDB/04152/2020
• Naslov: Centro de Investigação em Gestão de Informação (MagIC)/NOVA IMS

• Financer: ARIS - Javna agencija za znanstvenoraziskovalno in inovacijsko dejavnost Republike Slovenije
• Številka projekta: P2-0442
• Naslov: Podatkovne vede in digitalna preobrazba