Responsible AI, Einbindung in
komplexe IT-Umgebung, Security

“Bedenken und Bremsklötze” für den Einsatz von AI

Rainer Stropek

Passionierter Softwareentwickler, IT-Unternehmer, CoderDojo-Mentor, Lehrer

software architects gmbh
rainer@software-architects.at
https://rainerstropek.me

Threat
Scenarios

Out of scope for today

  • Challenges by AI for society (e.g. labor market)
  • AI-generated Phishing Attacks
  • Deepfake Scams and Impersonation
  • Misinformation and Spread of False Information
  • Automated cyberattacks leveraging AI
  • Social engineering aided by AI insights
  • AI-assisted insider threats
  • AI-driven market manipulation
  • Intellectual Property Theft (e.g., AI-assisted plagiarism)
  • ...

We focus on threat scenarios for specific AI-based software projects

Threat Scenarios

The usual cloud security stuff...

  • Data loss because of lack of strong data encryption (in transit, at rest)
  • Unauthorized data access because of missing/flawed auth
  • Unauthorized access because of secrets being in the wrong hands
  • Denial of service/wallet attacks
  • Attacks through code vulnerabilities
  • Cross-tenant data access because of poor tenant separation
  • GDPR violations (processing of PII in wrong regions)

Don't forget traditional software engineering and cloud security.

Your AI 🤖 is to a certain degree just another API/cloud app

Threat Scenarios

AI (LLM)-specific scenarios

  • Prompt injections, prompt escapes
  • Quality issues (prompts) after model updates
  • Evasion of content moderation or filters
  • Budget overrun
  • Incorrect/unwanted responses because of hallucinations, biases, etc.
  • Instabilities because of missing/flawed handling of token limits
  • Overblocking because of flawed content filtering
  • Unintended consequences of AI-generated content/decisions

New threats require new protective skills and tools.

Let's Protect Our 🤖 Friends

❗Secure hosting environment❗

  • Here: Focus on OpenAI
    • Self-hosting is not an option
  • Is using OpenAI-hosted services an option?
    • 👍 Access to the latest and greatest (preview) features
    • 👍 Cost-efficient price plans (enterprise-ready?)
    • 👍 GDPR-compliant DPA available for enterprise customers 🔗
    • 👎 Additional cloud provider (legal, billing, etc.)
    • 👎 Lack of integration into other hyperscaling clouds
  • Option: Host OpenAI models on Azure
    • 👍 Existing contracts with Azure
    • 👍 Integration with other Azure security features (e.g. VNets, AAD)
    • 👎 New features not immediately available after launched by OpenAI
    • 👎 Lack of features in Azure's ChatGPT app (UI), quality issues
  • DO use AAD-based authentication
    • DO use Managed Identity where possible, AVOID using shared secrets
  • DO favor PaaS and Serverless over IaaS/DIY
    • Benefit from built-in security measures
  • DO follow the principles of least privilege
    • Azure RBAC
  • CONSIDER limiting access to specific networks
    • CONSIDER VNet integration and Private Endpoints
  • DO use infrastructure-as-code
    • Repeatability, ability to audit, reviewability
  • DO apply the usual security stuff 😉
    • Policies, PIM, tenant separation, budget alerts, code and architecture reviews, static code analysis, logging, protect endpoints, etc.

❗Secure hosting environment❗

AI/LLM Specific Security

  • Be aware of Content Filtering 🔗
    • Consider using AI Content Safety 🔗
  • Carefully design and monitor the business model
    • Usage monitoring, token usage statistics
  • Use RAG pattern to provide verified/custom ground truth
    • Limit AI to the provided ground truth using system prompts
  • Monitor usage and adjust prompts if necessary
    • Consider using Prompt Flow for prompt design, testing, etc. 🔗
  • Don't let AI make legally binding/valid statements/decisions
    • Provide links to sources to end user (RAG)

In general: Only DIY if really necessary!

Key Takeaways

  • AI is a new and hot topic, but do not forget security fundamentals
    • Train your cloud security muscle 💪
  • You have to decide between OpenAI and Azure OpenAI
    • Leading edge vs. enterprise readiness
  • Prompting isn't trivial
    • Functionality, quality assurance
  • Avoid unfounded DIY mindset
    • Benefit from latest developments in APIs and AI-related cloud services

Thank you for your attention!