Industrial Feasibility of PEFTpyCoder: A Parameter-Efficient Fine-Tuned Model for In-House Code Assistance

We present an experience report on an industrial feasibility study that evaluates the application of code LLM technologies within an organization, ensuring that no data leaves the firm’s perimeter. To address a set of feasibility questions, we employed state-of-the-art techniques, including Low-Rank Adaptation (LoRA), Parameter-Efficient Fine-Tuning (PEFT), and 4-bit quantization, to develop a pipeline for creating a fine-tuned version of an existing code LLM that is small enough to allow both training and inference on commodity hardware, thereby eliminating the need for cloud-based solutions that would expose sensitive assets, such as parts of the existing code base, outside the firm’s local network. We evaluated the pipeline on a general programming language (namely Python) to assess its performance, and our results show that it is adequate for the task at hand. Other than concerns about data privacy, another motivation for this study stems from the firm’s adoption of a domain-specific programming language, which is not directly supported by most existing code LLMs. However, we believe that the lessons we learned in this experience could be valuable in many other contexts where a firm is evaluating the feasibility of an in-house AI-based solution rather than depending on external service providers.