Train your cake and eat it too! Repurposing collaborative training to tailor LLMs to private data without sharing

Radovič, Boris (avtor), Aljahdali, Mohammed (avtor), Canini, Marco (avtor), Pejović, Veljko (avtor), Khayyat, Zuhair (avtor)

In the emerging field of large language models (LLMs), a significant challenge arises when organizations with vast datasets lack the computational resources to independently train and fine-tune models. This issue stems from privacy, compliance, and resource constraints: organizations cannot share their sensitive data but still need external computational assistance for model training. In this paper, we implement, enhance, and empirically compare several methods, including Split Learning (SL) and select Federated Learning (FL) methods, which enable data-rich yet compute-poor clients to offload LLM training without sharing raw data. Our study evaluates these methods across multiple dimensions, including model quality and training time.

• Jezik: Angleški jezik
• Ključne besede: energy usage, digital data
• Vrsta gradiva: Drugo
• Tipologija: 1.08 - Objavljeni znanstveni prispevek na konferenci
• Organizacija: FRI - Fakulteta za računalništvo in informatiko
• Status publikacije: Objavljeno
• Različica publikacije: Objavljena publikacija
• Leto izida: 2024
• Št. strani: Str. 1-10
• PID: 20.500.12556/RUL-166792
• UDK: 004
• COBISS.SI-ID: 221660675

Avtorske pravice

Licenca Creative Commons je navedena na pristajalni strani članka (glej izvorni URL): "Licensed under Crative Commons Atribution 4.0 International". (Datum opombe 24. 1. 2025)

• Datum objave v RUL: 24.01.2025

Gradivo je del monografije

• Naslov: Workshop on Efficient Systems for Foundation Models II : ES-FoMo-II 2024
• Kraj izida: [Massachusetts
• Založnik: OpenReview
• Leto izida: 2024
• COBISS.SI-ID: 221659651

Licence

• Licenca: CC BY 4.0, Creative Commons Priznanje avtorstva 4.0 Mednarodna
• Povezava: http://creativecommons.org/licenses/by/4.0/deed.sl
• Opis: To je standardna licenca Creative Commons, ki daje uporabnikom največ možnosti za nadaljnjo uporabo dela, pri čemer morajo navesti avtorja.

Sekundarni jezik

• Jezik: Slovenski jezik
• Ključne besede: poraba energije, digitalni podatki