Deep Learning

Neural Networks 101

What is Machine Learning?

Using statistical models to

  • Understand data
  • Make predictions given more data
  • Make future actions given those predictions

Linear

Regression

Binary Classification

Clustering Algorithms

Some ML Algorithms/Models

Supervised

  • Logistic Regression
  • Linear Regression
  • Stochastic Gradient Descent
  • Support Vector Machines
  • K-Nearest Neighbors
  • Naive Bayes
  • Decision Trees/Forests
  • Neural Networks

Unsupervised

  • t-SNE
  • K-means
  • DBSCAN/OPTICS
  • Principal Component Analysis
  • Non-negative Matrix Factorization

What powers all of these?

VECTORS

What is a Neural Net?

It's like Neurons in the Body?

What's in a Hidden Layer?

What's a loss function?

How do we put this all together?

Input goes in

Output comes out

Backpropagation using gradient descent

Repeat

Hidden layers, Activation functions, and Loss Functions, Oh my!

Hidden layers

  • Convolution layers
  • Linear layers
  • Pooling layers
  • Recurrent layers
  • Normalization layers
  • Other layers that just do some fancy math stuff

Activation Functions

  • ReLU
  • L1/L2 Functions
  • Sigmoid

Loss Functions

  • Mean Squared Error
  • Cross-Entropy Loss
  • Kullback-Leibler Divergence Loss
  • Many more

COOL EXAMPLES

VGG16

Style Transfer (Adversarial Networks)

Style Transfer (Adversarial Networks)

Style Transfer (Adversarial Networks)

Style Transfer (Adversarial Networks)

Recurrent Neural Nets (RNNs)

Shakespeare

Recurrent Neural Nets (RNNs)

Wikipedia

// Article paragraph

Naturalism and decision for the majority of Arab countries' capitalide was grounded
by the Irish language by [[John Clair]], [[An Imperial Japanese Revolt]], associated 
with Guangzham's sovereignty. His generals were the powerful ruler of the Portugal 
in the [[Protestant Immineners]], which could be said to be directly in Cantonese 
Communication, which followed a ceremony and set inspired prison, training. The 
emperor travelled back to [[Antioch, Perth, October 25|21]] to note, the Kingdom 
of Costa Rica, unsuccessful fashioned the [[Thrales]], [[Cynth's Dajoard]], known 
in western [[Scotland]], near Italy to the conquest of India with the conflict. 
Copyright was the succession of independence in the slop of Syrian influence that 
was a famous German movement based on a more popular servicious, non-doctrinal 
and sexual power post. Many governments recognize the military housing of the 
[[Civil Liberalization and Infantry Resolution 265 National Party in Hungary]], 
that is sympathetic to be to the [[Punjab Resolution]]
(PJS)[http://www.humah.yahoo.com/guardian.
cfm/7754800786d17551963s89.htm Official economics Adjoint for the Nazism, Montgomery 
was swear to advance to the resources for those Socialism's rule, 
was starting to signing a major tripad of aid exile.]]



// Markdown example

{ { cite journal | id=Cerling Nonforest Department|format=Newlymeslated|none } }
''www.e-complete''.

'''See also''': [[List of ethical consent processing]]

== See also ==
*[[Iender dome of the ED]]
*[[Anti-autism]]

===[[Religion|Religion]]===
*[[French Writings]]
*[[Maria]]
*[[Revelation]]
*[[Mount Agamul]]

== External links==
* [http://www.biblegateway.nih.gov/entrepre/ Website of the World Festival. The labour of India-county defeats at the Ripper of California Road.]

==External links==
* [http://www.romanology.com/ Constitution of the Netherlands and Hispanic Competition for Bilabial and Commonwealth Industry (Republican Constitution of the Extent of the Netherlands)]



// XML Example

<page>
  <title>Antichrist</title>
  <id>865</id>
  <revision>
    <id>15900676</id>
    <timestamp>2002-08-03T18:14:12Z</timestamp>
    <contributor>
      <username>Paris</username>
      <id>23</id>
    </contributor>
    <minor />
    <comment>Automated conversion</comment>
    <text xml:space="preserve">#REDIRECT [[Christianity]]</text>
  </revision>
</page>

Recurrent Neural Nets (RNNs)

LaTeX (Math Typing)

Recurrent Neural Nets (RNNs)

LaTeX (Math Typing)

Recurrent Neural Nets (RNNs)

Linux Source Code (in C)

/*
 * Increment the size file of the new incorrect UI_FILTER group information
 * of the size generatively.
 */
static int indicate_policy(void)
{
  int error;
  if (fd == MARN_EPT) {
    /*
     * The kernel blank will coeld it to userspace.
     */
    if (ss->segment < mem_total)
      unblock_graph_and_set_blocked();
    else
      ret = 1;
    goto bail;
  }
  segaddr = in_SB(in.addr);
  selector = seg / 16;
  setup_works = true;
  for (i = 0; i < blocks; i++) {
    seq = buf[i++];
    bpf = bd->bd.next + i * search;
    if (fd) {
      current = blocked;
    }
  }
  rw->name = "Getjbbregs";
  bprm_self_clearl(&iv->version);
  regs->new = blocks[(BPF_STATS << info->historidac)] | PFMR_CLOBATHINC_SECONDS << 12;
  return segtable;
}





/*
 * If this error is set, we will need anything right after that BSD.
 */
static void action_new_function(struct s_stat_info *wb)
{
  unsigned long flags;
  int lel_idx_bit = e->edd, *sys & ~((unsigned long) *FIRST_COMPAT);
  buf[0] = 0xFFFFFFFF & (bit << 4);
  min(inc, slist->bytes);
  printk(KERN_WARNING "Memory allocated %02x/%02x, "
    "original MLL instead\n"),
    min(min(multi_run - s->len, max) * num_data_in),
    frame_pos, sz + first_seg);
  div_u64_w(val, inb_p);
  spin_unlock(&disk->queue_lock);
  mutex_unlock(&s->sock->mutex);
  mutex_unlock(&func->mutex);
  return disassemble(info->pending_bh);
}

static void num_serial_settings(struct tty_struct *tty)
{
  if (tty == tty)
    disable_single_st_p(dev);
  pci_disable_spool(port);
  return 0;
}

static void do_command(struct seq_file *m, void *v)
{
  int column = 32 << (cmd[2] & 0x80);
  if (state)
    cmd = (int)(int_state ^ (in_8(&ch->ch_flags) & Cmd) ? 2 : 1);
  else
    seq = 1;
  for (i = 0; i < 16; i++) {
    if (k & (1 << 1))
      pipe = (in_use & UMXTHREAD_UNCCA) +
        ((count & 0x00000000fffffff8) & 0x000000f) << 8;
    if (count == 0)
      sub(pid, ppc_md.kexec_handle, 0x20000000);
    pipe_set_bytes(i, 0);
  }
  /* Free our user pages pointer to place camera if all dash */
  subsystem_info = &of_changes[PAGE_SIZE];
  rek_controls(offset, idx, &soffset);
  /* Now we want to deliberately put it to device */
  control_check_polarity(&context, val, 0);
  for (i = 0; i < COUNTER; i++)
    seq_puts(s, "policy ");
}




/*
 *  Copyright (c) 2006-2010, Intel Mobile Communications.  All rights reserved.
 *
 *   This program is free software; you can redistribute it and/or modify it
 * under the terms of the GNU General Public License version 2 as published by
 * the Free Software Foundation.
 *
 *        This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 *
 *  GNU General Public License for more details.
 *
 *   You should have received a copy of the GNU General Public License
 *    along with this program; if not, write to the Free Software Foundation,
 *  Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 */

#include <linux/kexec.h>
#include <linux/errno.h>
#include <linux/io.h>
#include <linux/platform_device.h>
#include <linux/multi.h>
#include <linux/ckevent.h>

#include <asm/io.h>
#include <asm/prom.h>
#include <asm/e820.h>
#include <asm/system_info.h>
#include <asm/setew.h>
#include <asm/pgproto.h>

#define REG_PG    vesa_slot_addr_pack
#define PFM_NOCOMP  AFSR(0, load)
#define STACK_DDR(type)     (func)

#define SWAP_ALLOCATE(nr)     (e)
#define emulate_sigs()  arch_get_unaligned_child()
#define access_rw(TST)  asm volatile("movd %%esp, %0, %3" : : "r" (0));   \
  if (__type & DO_READ)

static void stat_PC_SEC __read_mostly offsetof(struct seq_argsqueue, \
          pC>[1]);

static void
os_prefix(unsigned long sys)
{
#ifdef CONFIG_PREEMPT
  PUT_PARAM_RAID(2, sel) = get_state_state();
  set_pid_sum((unsigned long)state, current_state_str(),
           (unsigned long)-1->lr_full; low;
}

GPT-2

https://app.inferkit.com/demo

GANs (Generative Adversarial Networks)

https://thispersondoesnotexist.com/

https://thisxdoesnotexist.com/

GANs (Generative Adversarial Networks)

Tesla's Self Driving

Reinforcement Learning

TOO MANY EXAMPLES!!!

  • Composing music (And generating a MIDI file
  • Translation
  • Learning to play chess or other much more complicated games
  • Handwriting Generation (I did this :D)
  • Sports predictions
  • Voice recognition
  • Financial market analysis/predictions
  • Image colorization (add color to old B/W photos)
  • GitHub CoPilot
  • SO MUCH MORE!

Conclusion

Cool/Interesting Links

  • https://mind.cs.byu.edu/courses/474/schedule.php (BYU Class)
  • https://image-net.org/ (Massive Image Classification Data set)
  • http://playground.tensorflow.org/ (Tensorflow playground)
  • https://thisxdoesnotexist.com/ (Lots of GANs)
  • https://github.com/vdumoulin/conv_arithmetic (CNN Animations)
  • http://neuralnetworksanddeeplearning.com/chap2.html (Backprop)
  • https://jalammar.github.io/ (Tons of awesome articles on DL)
  • https://jalammar.github.io/feedforward-neural-networks-visual-interactive/ (Basic NN Article)
  • https://www.fritz.ai/style-transfer/ (Style Transfer)
  • And a ton more, most linked from that first BYU class schedule page

Deep Learning 101

By Ethan Brouwer

Deep Learning 101

  • 78