Introduction

Numerical Methods

David Mayerich

Scalable Tissue Imaging and Modeling (STIM) Laboratory

Department of Electrical and Computer Engineering

Cullen College of Engineering

University of Houston

David Mayerich

STIM Laboratory, University of Houston

Goals and Expectations

Fundamentals of numerical methods
- Resolving numerical errors: quantization, binarization, etc.
- Solving complex equations and evaluating functions
- Linear systems
- Partial differential equations and physical systems

David Mayerich

STIM Laboratory, University of Houston

Experience in implementation (C/C++ and MATLAB/Python)

When you encounter a numerical problem, you should know how to find the best (fastest, most accurate, most efficient, etc.) solution

Grades should be predictable: rubrics are provided for all assignments

Resources

Numerical methods in Practice: A Programming Guide to Numerical Computing Using C and Python, D. Mayerich
- open-access textbook written specifically for this class
- https://stim.ee.uh.edu/education/ece-3340-numerical-methods/

David Mayerich

STIM Laboratory, University of Houston

Numerical Methods for Scientists and Engineers, R.W. Hamming

Math-focused discussion
Short and precise
Inexpensive

Numerical Recipes: The Art of Scientific Computing, W.H. Press, et al.

C implementations
Advanced problems
Expensive

Alternatives

Prerequisites

David Mayerich

STIM Laboratory, University of Houston

C/C++ programming
MATLAB/Python

a \in \mathbb{R} \quad \mathbf{b} \in \mathbb{C}^3 \quad \mathbf{A}\in\mathbb{R}^{4\times 3}

Taylor and Maclaurin series

e^x \approx 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!}

Linear systems

\mathbf{A}\mathbf{X}=\mathbf{B} \quad \mathbf{A}\in\mathbb{C}^{3\times 3}

Partial differential equations

\frac{\partial\phi(\mathbf{x},t)}{\partial t} = \nabla \cdot \left[ D(\phi, \mathbf{x}) \nabla \phi(\mathbf{x}, t) \right]

Taylor and Maclaurin series

MATH 3321 - Engineering Math

ECE 3331 - Programming Apps.

Grading

5 % Homework

David Mayerich

STIM Laboratory, University of Houston

10% Quizzes

20% Exams 1 & 2

25% Programming Assignments

30% Final Exam

Weekly assignments that reflect material on the exams

Homework review, sample exam questions

Very short (ex. 1 question), every 1-2 days

Mid-term exams (closed book, closed note)

Implement numerical solutions to mathematical problems - largest will involve partial differential equations

Previous material + differentiation, integration, partial differential equations, interpolation, fitting functions

Homework Assignments

5% of your final grade – primarily practice for the exams

David Mayerich

STIM Laboratory, University of Houston

Grade based on written homework
- Taken from class lectures and textbook
- You may work together, but don't had in copied work
- I will review for completion and as a benchmark for class performance
- Submit a PDF on Canvas

I will only review assigned homework, however there are other practice problems in the textbook

Teams: feel free to ask and answer questions about the homework

Quizzes

Test your understanding of the material with regular quizzes
My aim here is to make sure you can predict your performance on the exams
Main goal here is to address a recent trend of students leaning on LLMs: I've encountered too many cases where students don't realize that they don't understand the material and do poorly on exams.
Calculators: you'll want a scientific calculator (same rules as the ACT)
Generally be 1-2 simple questions at the beginning of class
Submit via Canvas or just hand it in

David Mayerich

STIM Laboratory, University of Houston

Exams

Exams are comprehensive - previous material is fair game
- 20% numerical representations, algorithms, loss of precision
- 20% linear systems, tensors, and tensor operations
- 30% fitting, interpolation, calculus

David Mayerich

STIM Laboratory, University of Houston

Exam questions rely on material from homework, quizzes, lectures, textbook
There will be some programming questions but I won't rigorously review "code"
Calculators: you'll want a scientific calculator (same rules as the ACT)

Teams: don't ask/answer test-related questions until I confirm all exams are completed
(since some students may be required to take the exam outside of class time)

Programming Assignments

25% of your grade - work independently

David Mayerich

STIM Laboratory, University of Houston

Code must compile using gcc on the ECE server
- If your code doesn't work, output what you did complete
- The code must still compile
- Comment frequently - partial credit relies on me understanding your code

Why learn about numerical methods?

Implementation depends on requirements and constraints
- High-level applications can use MATLAB and external libraries
- Low-level applications are under pressure: memory, speed, processing power

David Mayerich

STIM Laboratory, University of Houston

Most low-level implementations are simple
- Algorithms are well-understood, but we will discuss recent advances
- Implementation depends platform and therefore still challenging
- Constrained by architecture, memory, precision, speed, discretization errors

Examples of algorithms that pose challenges:
- subtraction
- evaluating non-polynomial functions (sine, cosine, exp, logarithms)
- integration and differentiation
- optimization (finding minima and maxima) - often used in machine learning

Topics

1. Mathematics - notation that will help us communicate

2. Programming - translating from math to a computer (via C)

3. Discrete Math - theory for evaluating algorithms

4. Errors - how and where do computers make mistakes?

5. Numbers - how values are stored and manipulated in digital systems

6. Functions - use algorithms and series to evaluate special functions

7. Solving Equations - numerical methods to solve equations with special functions

David Mayerich

STIM Laboratory, University of Houston

Exam 1

Topics

12. Regression - building functions from samples

13. Differential Equations - numerical techniques for solving complex physical systems

David Mayerich

STIM Laboratory, University of Houston

Final Exam

8. Linear Systems - mathematical operations that take a lot of time

9. Matrix Properties - assessing linear systems and how accurately they can be solved

10. Random Numbers - probability and generating random numbers

11. Calculus - differentiation and integration using functions and values in digital systems

Exam 2

Resources

Website:
http://stim.ee.uh.edu

David Mayerich

STIM Laboratory, University of Houston

Teams forum - please ask questions:
- homework, textbook problems, and lecture slides
- exam questions AFTER the exams have been graded
- conceptual questions about programming assignments - don't post blocks of code
- messages can be sent to me privately via Teams