Intelligent Apps with Angular and Azure AI Search

ngSummit Rome | December 2023

Natalia Venditto

microfrontend.dev - @anfibiacreativa

Image credit DALL-E3

Once upon a time...

© Austin Neil | Unsplash.com

I build
e2e cloud applications!

(Like 3 months ago)

© Ivana Cajina | Unsplash.com

© Ivana Cajina | Unsplash.com

RAG?

Natalia Venditto

aka anfibiacreativa

Principal JavaScript e2e DX Lead  @Microsoft Azure

Google Developer Expert for Angular and Web Technologies

Author of https://microfrontend.dev

2021 Microsoft Most Valuable Professional

What's in it for me?

Retrieval Augmented Generation

Model architecture used in natural language processing (NLP) that combines retrieval-based and generative approaches.

microfrontend.dev - @anfibiacreativa

DATASET

Response

Retriever

SELECTS RELAVANT PASSAGES

FORMULATES RESPONSE

microfrontend.dev - @anfibiacreativa

DATASET

Response

Retriever

SELECTS RELAVANT PASSAGES

FORMULATES RESPONSE

microfrontend.dev - @anfibiacreativa

Am I now ready to build RAG apps?

© Mahdis Musavi | Unsplash.com

microfrontend.dev - @anfibiacreativa

Lorem ipsum

Lorem ipsum

Lorem ipsum

HOW WE BUILD IT?

Specification, contracts, architecture decisions (like patterns).

Lorem ipsum

Lorem ipsum

WITH WHAT WE BUILD IT?

Tech stack, integrations (using best practices).

WHERE WE RUN IT?

Infra and pipelines.

REFERENCE

AI Models

Or how things are shaped.

microfrontend.dev - @anfibiacreativa

WITH WHAT?

GPT and LLMs

Generative Pre-trained (on massive amounts of data) Transformers: a prompt-based model that learns to predict the next word in a sequence.

It is autoregressive (one token at a time)*

microfrontend.dev - @anfibiacreativa

GPT

microfrontend.dev - @anfibiacreativa

LLM

Large Language Models, (encompasses the GPT category), and are large because they can include up to trillions of parameters (weight and biases) that establish relationships.

GPT 3.5, GPT 4, TURBO

WITH WHAT?

Completions

Completions are the process of predicting the completion of a sequence (of text) based on a given input.

 

We can think of code completion with Copilot!

microfrontend.dev - @anfibiacreativa

Embeddings

Embeddings are the vectorial representation of words, phrases, sentences, etc, in the continuous vector space.

 

WORDS mapped to their VECTOR representation, so we can operate with them. Semantically similar words will be close together in vector space.

microfrontend.dev - @anfibiacreativa

SEMANTIC RANKER CAPTIONS

RETRIEVAL MODE

APPROACH

Organization of the words depending on the semantic similarity, and generation of textual descriptions.

microfrontend.dev - @anfibiacreativa

Establishes the flow of retrieval and generation, for example 'retrieve then read'.

Defines whether retrieval is based on text similarity or vector representation.

TEMPERATURE

Controls randomness.

Azure OpenAI SDK

Client library to implement Open AI services on Azure.

microfrontend.dev - @anfibiacreativa

For JavaScript!

WITH WHAT?

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

/**
 * Demonstrates how to get chat completions for a chat context.
 *
 * @summary get chat completions.
 */

const { OpenAIClient, AzureKeyCredential } = require("@azure/openai");

// Load the .env file if it exists
require("dotenv").config();

// You will need to set these environment variables or edit the following values
const endpoint = process.env["ENDPOINT"] || "<endpoint>";
const azureApiKey = process.env["AZURE_API_KEY"] || "<api key>";

const messages = [
  { role: "system", content: "You are a helpful assistant. You will talk like a pirate." },
  { role: "user", content: "Can you help me?" },
];

async function main() {

  const client = new OpenAIClient(endpoint, new AzureKeyCredential(azureApiKey));
  const deploymentId = "gpt-35-turbo";
  const result = await client.getChatCompletions(deploymentId, messages);

  for (const choice of result.choices) {
    console.log(choice.message);
  }
}

main().catch((err) => {
  console.error("The sample encountered an error:", err);
});

module.exports = { main };
# Chat with the bot
# Chat-app-protocol

POST {{api_host}}/chat
Content-Type: application/json

{
  "messages": [{
    "content": "How to search and book rentals?",
    "role": "user"
  }],
  "context": {
    "approach":"rrr",
    "retrieval_mode": "hybrid",
    "semantic_ranker": true,
    "semantic_captions": false,
    "top": 3,
    "suggest_followup_questions": false
  }
}

###

Swappable

The Chat App Protocol as a contract between frontends and backends using the same services.

microfrontend.dev - @anfibiacreativa

backends

microfrontend.dev - @anfibiacreativa

HOW?

Application Architecture

Web components that can be bootstrapped anywhere.

microfrontend.dev - @anfibiacreativa

HOW?

SEARCH SERVICE

BLOB

Frontend

microfrontend.dev - @anfibiacreativa

AZURE AI SEARCH

Repo

INDEXER SERVICE

AZURE OPEN AI SERVICE

SEARCH SERVICE

BLOB

Frontend

microfrontend.dev - @anfibiacreativa

AZURE AI SEARCH

Repo

INDEXER SERVICE

AZURE OPEN AI SERVICE

HOW?

Sources.

Data ingestion and indexing.

microfrontend.dev - @anfibiacreativa

import { type SearchIndex } from '@azure/search-documents';
import { encoding_for_model, type TiktokenModel } from '@dqbd/tiktoken';
import { type AzureClients } from '../plugins/azure.js';
import { type OpenAiService } from '../plugins/openai.js';

// other code

export class Indexer {
  private blobStorage: BlobStorage;

  constructor(
    private logger: BaseLogger,
    private azure: AzureClients,
    private openai: OpenAiService,
    private embeddingModelName: string = 'text-embedding-ada-002',
  ) {
    this.blobStorage = new BlobStorage(logger, azure);
  }

  async createSearchIndex(indexName: string) {
    this.logger.debug(`Ensuring search index "${indexName}" exists`);

    const searchIndexClient = this.azure.searchIndex;

    const names: string[] = [];
    const indexNames = await searchIndexClient.listIndexes();
    for await (const index of indexNames) {
      names.push(index.name);
    }
    
    // mode code

FINE-TUNING WITH LANGCHAIN 🦜️🔗

https://js.langchain.com/docs/get_started/introduction

https://langchain.com/docs/integrations/platforms/microsoft

microfrontend.dev - @anfibiacreativa

WITH WHAT?

Frontend

microfrontend.dev - @anfibiacreativa

WITH WHAT?

<!-- Other code -->
  <mat-sidenav-content>
    <div class="inner-wrapper">
      <button class="button__button settings" title="Application Settings" (click)="sidenav.toggle()">
        <img src="./assets/svg/gear-solid.svg?raw" alt="Settings" class="icon__img" />
        <span>{{ settingsDefaults.panelLabel }}</span>
      </button>

      <chat-component
        [title]="title"
        [attr.data-input-position]="inputPosition"
        [attr.data-interaction-model]="interactionModel"
        data-api-url=""
        [attr.data-use-stream]="streaming"
        [attr.data-approach]="approach"
        [attr.data-overrides]="overrides"
      ></chat-component>
    </div>
<!-- Other code -->
// other code
@customElement('chat-component')
export class ChatComponent extends LitElement {
  //--
  // Public attributes
  // --

  @property({ type: String, attribute: 'data-input-position' })
  inputPosition = 'sticky';

  @property({ type: String, attribute: 'data-interaction-model' })
  interactionModel: 'ask' | 'chat' = 'chat';

  @property({ type: String, attribute: 'data-api-url' })
  apiUrl = chatHttpOptions.url;

  @property({ type: String, attribute: 'data-use-stream', converter: (value) => value?.toLowerCase() === 'true' })
  useStream: boolean = chatHttpOptions.stream;

  @property({ type: String, attribute: 'data-overrides', converter: (value) => JSON.parse(value || '{}') })
  overrides: RequestOverrides = {};
  //
  // more code

Frontend

microfrontend.dev - @anfibiacreativa

window.postMessage(m, tO, t)

WITH WHAT?


  // other code
  
  @HostListener('window:message',['$event'])
  onMessage(event: Event) {
    console.log('I hear you chat-component!');
    // Do something here
    // For example, make sure opened drawers and expanded items don't overlap
  }
  
  // more code

Go to demo!

Streaming ndJSON

ndJSON or newline delimited JSON, is a data interchange format that represents JSON objects in a file separated by newline characters.

Each line is a valid JSON object, and so is the whole file.

microfrontend.dev - @anfibiacreativa

import { NdJsonParserStream } from './data-format/ndjson.js';
import { globalConfig } from '../../config/global-config.js';

export function createReader(responseBody: ReadableStream<Uint8Array> | null) {
  return responseBody?.pipeThrough(new TextDecoderStream()).pipeThrough(new NdJsonParserStream()).getReader();
}

export async function* readStream<T>(reader: any): AsyncGenerator<T, void> {
  if (!reader) {
    throw new Error('No response body or body is not readable');
  }

  let value: JSON | undefined;
  let done: boolean;
  while ((({ value, done } = await reader.read()), !done)) {
    yield new Promise<T>((resolve) => {
      setTimeout(() => {
        resolve(value as T);
      }, globalConfig.BOT_TYPING_EFFECT_INTERVAL);
    });
  }
}

// Stop stream
export function cancelStream<T>(stream: ReadableStream<T> | null): void {
  if (stream) {
    stream.cancel();
  }
}
// more code
// this is the backend service
async *runWithStreaming(
    messages: Message[],
    context?: ChatApproachContext,
  ): AsyncGenerator<ApproachResponseChunk, void> {
    const { completionRequest, dataPoints, thoughts } = await this.baseRun(messages, context);
    const openAiChat = await this.openai.getChat();
    const chatCompletion = await openAiChat.completions.create({
      ...completionRequest,
      stream: true,
    });
    let id = 0;
    for await (const chunk of chatCompletion) {
      const responseChunk = {
        choices: [
          {
            index: 0,
            delta: {
              content: chunk.choices[0].delta.content ?? '',
              role: 'assistant' as const,
              context: {
                data_points: id === 0 ? dataPoints : undefined,
                thoughts: id === 0 ? thoughts : undefined,
              },
            },
            finish_reason: chunk.choices[0].finish_reason,
          },
        ],
        object: 'chat.completion.chunk' as const,
      };
      yield responseChunk;
      id++;
    }
  }

CHALLENGE!

Deployment

All running software needs infra.

microfrontend.dev - @anfibiacreativa

WHERE TO?

See video!

Development

Specification

Setup

Design

Integration

Delivery

Operations

DEVELOPMENT

DESIGN

DEPLOYMENT

DEV

OPS

ARQ

microfrontend.dev - @anfibiacreativa

Ethics

"When we implement AI responsibly, we can bring tremendous positive impact for individuals, industries, and society."

microfrontend.dev - @anfibiacreativa

From our own training.

WITH WHAT?

Thank you!

@anfibiacreativa - https://www.microfrontend.dev

Learn more

All images except those that credited to Unsplash and respective author, were generated with Bing Image Generator.

Intelligent Apps with Angular and Azure OpenAI

By Natalia Venditto

Intelligent Apps with Angular and Azure OpenAI

  • 82