Safe Financial AI with Claude 2 and AWS Bedrock

 

Richard Whaling

WindyCityDevFest

Oct 24, 2023

About Me

  • I'm Richard, he/they
  • NLP in grad school
    • (I taught ancient Greek to Markov chains)
  • Consulted, wrote a book about Scala Native
  • Sr. Director at M1
  • Data Pipelines/Warehouse/BI Platforms
  • Banking and Brokerage
  • ML Eng.
  • AI Safety

This Talk

  • Been looking at LLM + Time Series for a few months because:
    • My team found that our models understood tables from our wikis, even with markup stripped
    • This research paper from AWS:
      • Yu et al., "Temporal Data Meets LLM: Explainable Financial Time Series Forecasting"
      • https://arxiv.org/pdf/2306.11025.pdf
    • LLM's can outperform SOTA gradient-boosted trees, and explain their reasoning
  • This is exciting and scary

AI Safety

Three high-level categories of risks that scare me:

  • Hallucination
    • Pretty well understood, but not solved
  • Security/Data Exfiltration
    • Providers often keep data and use it for training
    • Recovering training data from models is very real
  • Legal/Regulatory
    • Liability/IP
    • As a broker-dealer, M1 cannot provide investment advice

Security with Bedrock

  • Recently GA, securely hosted foundation models
  • GDPR and HIPAA compliant
  • Both proprietary and open models
    • Anthropic, AI21, Cohere
    • Meta, Stability
    • Amazon's Titan models
  • Does not store or retain prompts or responses
  • Supports KMS, PrivateLink, etc.
import boto3
import json

bedrock = boto3.client('bedrock-runtime', 'us-west-2', 
          endpoint_url='https://bedrock-runtime.us-west-2.amazonaws.com')

modelId = 'anthropic.claude-v2'
accept = 'application/json'
contentType = 'application/json'

def complete(prompt, tokens=1000, temperature=0.5):
    claude_prompt = f"\n\nHuman:{prompt}\n\nAssistant:"
    body = json.dumps({
                        "prompt": claude_prompt,
                        "temperature": 0.5,
                        "top_p": 1,
                        "top_k": 250,
                        "max_tokens_to_sample": 1000,
                        "stop_sequences": ["\n\nHuman:"]
                     })
    response = bedrock.invoke_model(body=body, modelId=modelId, 
                                    accept=accept, contentType=contentType)
    return json.loads(response['body'].read())

Now What?

That's all the slides, switching to live code

  1. Simple prompt, proof-of-concept
  2. Add data until the prompt breaks
  3. Fix issues by dividing the problem
    1. Overall portfolio performance
    2. Extracting outliers
    3. Drill down security analysis
    4. Summarization

Safe Financial AI

By Richard Whaling

Made with Slides.com

Safe Financial AI

  • 37

More from Richard Whaling