As Large Language Models (LLMs) increasingly support critical sectors such as healthcare, finance, and public governance, ensuring data confidentiality and robust access control is a pressing societal challenge. Traditional security mechanisms isolate sensitive resources from unauthorized users, yet existing LLM safety approaches often fail to enforce strict segregation of confidential data. In this work, we introduce SecureLLM, a novel compositional framework for building secure LLMs that integrates fine-tuning with traditional access security measures to protect private information. By fine-tuning LLMs on segregated, “siloed” training data and composing their outputs at inference time based solely on a user’s verified credentials, SecureLLM not only prevents unauthorized data leakage but also enables accurate responses for complex queries spanning multiple data silos. Our method is demonstrated on a challenging natural-language-to-SQL translation task and is designed with real-world applications in mind, where protecting sensitive information is critical.