Fortifying OpenAI Deployments in Microsoft Azure

Introduction

Always prefer IaC over manual maintenance of cloud artifacts
- Native in Azure: Bicep
- OSS and 3rd party options (e.g. Terraform, Pulumi, etc.)
IaC is a security measure
- Enables code reviews
- Repeatable (e.g. for staging)
- IaC combined with source control ➡️ auditability
- Good practices written in code can be shared
IaC only leads to enhanced security if access to Azure control plane is properly secured!
- Investment in AAD and RBAC is required for that
- MFA, PIM, Conditional Access, secured deployment processes, etc.

OpenAI Access Key is highly sensitive data
- ⚠️ Modern OpenAI APIs persist data
- Economic Denial of Sustainability (EDoS)
Where to store secrets for M2M communication?
- How to create those secrets?
- How to regularly update those secrets?
- Secrets in the hand of admins circumvent MFA
The fact that code is running in a specific Azure service is proof of identity
- Uses developer identity (MFA) for local debugging
- Managed Identity can get rid of most secrets for OpenAI inside Azure

Azure supports virtual networks (VNets)
- Lots of security components available for VNets
- E.g. NSGs, Azure Firewall, App Gateway, Frontdoor, etc.
Most PaaS offerings cannot be moved into VNets
- They are run by Microsoft
Solution: Private Endpoints
- Enables accessing PaaS services over PE in your own VNet
- PaaS services does not need to be available on public Internet
- Available for many Azure PaaS offerings 🔗