{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of FINAL Template_for CIS521_midterm notes.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "brNDGX-kvzds"
      },
      "source": [
        "# Template for Notes for CIS 521 Midterm 1\n",
        "\n",
        "The midterm exam for CIS 521 will be **open notes**. No other matierals are allowed for the exam - you may not use the textbook, the internet, a Python envionement, or communicate with anyone during the exam. \n",
        "\n",
        "This python notebook is where you should save all of your notes.  It provides a structured template that allows you to add summaries for each of the textbook sections that are part of the required reading, plus room for lecture notes and definitions.\n",
        "\n",
        "Your summaries must be written in your own words.  Do not copy and paste from the textbook.  Do not share your notes with classmates, or copy any of your classmates' notes.  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aPtM8g7R0Pds"
      },
      "source": [
        "# Instructions\n",
        "\n",
        "1. Make a copy of this Google Colab file. From the colab menu, pick **File > Save a copy in Drive**.  That will save a copy of this notebook in your Google drive.\n",
        "2. Go through each section of the notebook, and fill in all the summaries for each chapter and its corresponding sections.  Be sure to save your work frequently!\n",
        "3. Read the notes below about how you can format your summaries.\n",
        "4. In addition to these notes, you are allowed 1 page (both sides) for figures or handwritten notes.\n",
        "5. Submit your version of this notebook to Gradescope before you start the exam.\n",
        "6. If you are taking the exam in person, then you must print a copy of your notes for yourself.   Here's how to print: \n",
        "* Run the last cell of this notebook\n",
        "* Click the \"Mirror cell in tab\" button (this is located in the upper right corner of the cell, immediately to the right of the trash can).  This will open the cell.  \n",
        "* In Chrome, you can then print your notes using the **File > Print** option from the browser's menu.  \n",
        "* In the print options, change the layout from Landscape to Portrait. \n",
        "* Clicking the Save, will save a PDF that you can then print.\n",
        "\n",
        "## Formatting Notes\n",
        "\n",
        "1. The max length for each section's summary is 500 words / 3000 characters.\n",
        "2. You can also write a 2000 word summary of the lectures for each module.\n",
        "3. You may also write definitions of concepts for each module.  The definitions should be written in your own words (not copy and pasted), and they should be no more than 20 words long for each concept. \n",
        "2. You can use Markdown formatting for your summaries. \n",
        "3. You can use Latex formatting for equations. Put two dollar signs before and after your equation like this: \n",
        "$$ \\epsilon = \\frac{1}{mc^2} $$\n",
        "4. When you write python code or psuedocode algorithms, you should surround it with triple backticks like this:\n",
        "```\n",
        "def function():\n",
        "      print('hello world')\n",
        "```\n",
        "5. Write \\ whenever you want to keep writing on the next line without a newline being inserted."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cw5IpDLCZa4W"
      },
      "source": [
        "## Example summary\n",
        "\n",
        "This is a sample summary of [Section 2.1.1](https://web.stanford.edu/~jurafsky/slp3/2.pdf ) of Jurfasky and Martin's Speech and Language Processing's book\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lzaavW96tcxM",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "707e2c74-4fde-4313-9685-73102cafd50a"
      },
      "source": [
        "chapter_2_1_example = \"\"\"\n",
        "Regular expressions are an algebraic notation for characterizing \\\n",
        "a set of strings that are useful for searching for a pattern or through a corpus.\\\n",
        "Only the first matching result will be shown. The search string is case sensitive, \\\n",
        "but it can consist of a single character or a sequence of characters. \\\n",
        "While square braces specify a disjunction of characters, the dash can more easily look \\\n",
        "for characters over a range. The caret specifies the non-existence of a character. \\\n",
        "Additionally, the (?) is used to for flexibility on a single character, meaning \\\n",
        "\"the preceding character or nothing\". More importantly, the Kleene star means zero or \\\n",
        "more occurrences of the immediately previous character or regular expression in square braces.\\\n",
        "...\n",
        "\"\"\"\n",
        "\n",
        "length = len(chapter_2_1_example.split(\" \"))\n",
        "print(\"The example summary is {len} words long, so you could write an additional \\\n",
        "{remaining} words.\".format(len=length, remaining=(500-length)))"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The example summary is 116 words long, so you could write an additional 384 words.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s3Uk9gQTevcj"
      },
      "source": [
        "## Example summary length\n",
        "\n",
        "The maximum summary length for each section is 500 words / 3000 characters.  Hopefully you'll find that to be quite generous. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8DvZ57A-djo1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "812b43ca-eab0-424f-b8a0-8ddb26375576"
      },
      "source": [
        "# This is an example of a 500 word placeholder\n",
        "placeholder = \"\"\"ultrices vitae auctor eu augue ut lectus arcu bibendum at varius \\\n",
        " vel pharetra vel turpis nunc eget lorem dolor sed viverra ipsum nunc aliquet \\\n",
        " bibendum enim facilisis gravida neque convallis a cras semper auctor neque vitae \\\n",
        " tempus quam pellentesque nec nam aliquam sem et tortor consequat id porta nibh \\\n",
        " venenatis cras sed felis eget velit aliquet sagittis id consectetur purus ut \\\n",
        " faucibus pulvinar elementum integer enim neque volutpat ac tincidunt vitae semper \\ \n",
        " quis lectus nulla at volutpat diam ut venenatis tellus in metus vulputate eu \\\n",
        " scelerisque felis imperdiet proin fermentum leo vel orci porta non pulvinar neque \\ \n",
        " laoreet suspendisse interdum consectetur libero id faucibus nisl tincidunt eget \\\n",
        " nullam non nisi est sit amet facilisis magna etiam tempor orci eu lobortis \\\n",
        " elementum nibh tellus molestie nunc non blandit massa enim nec dui nunc mattis \\\n",
        " enim ut tellus elementum sagittis vitae et leo duis ut diam quam nulla porttitor \\\n",
        " massa id neque aliquam vestibulum morbi blandit cursus risus at ultrices mi tempus \\\n",
        " imperdiet nulla malesuada pellentesque elit eget gravida cum sociis natoque \\\n",
        " penatibus et magnis dis parturient montes nascetur ridiculus mus mauris vitae \\\n",
        " ultricies leo integer malesuada nunc vel risus commodo viverra maecenas accumsan \\\n",
        " lacus vel facilisis volutpat est velit egestas dui id ornare arcu odio ut sem nulla \\\n",
        " pharetra diam sit amet nisl suscipit adipiscing bibendum est ultricies integer \\\n",
        " quis auctor elit sed vulputate mi sit amet mauris commodo quis imperdiet massa \\\n",
        " tincidunt nunc pulvinar sapien et ligula ullamcorper malesuada proin libero nunc \\\n",
        " consequat interdum varius sit amet mattis vulputate enim nulla aliquet porttitor \\\n",
        " lacus luctus accumsan tortor posuere ac ut consequat semper viverra nam libero \\\n",
        " justo laoreet sit amet cursus sit amet dictum sit amet justo donec enim diam \\\n",
        " vulputate ut pharetra sit amet aliquam id diam maecenas ultricies mi eget mauris \\\n",
        " pharetra et ultrices neque ornare aenean euismod elementum nisi quis eleifend quam \\\n",
        " adipiscing vitae proin sagittis nisl rhoncus mattis rhoncus urna neque viverra justo \\\n",
        " nec ultrices dui sapien eget mi proin sed libero enim sed faucibus turpis in \\\n",
        " eu mi bibendum neque egestas congue quisque egestas diam in arcu cursus euismod \\\n",
        " quis viverra nibh cras pulvinar mattis nunc sed blandit libero volutpat sed cras \\\n",
        " ornare arcu dui vivamus arcu felis bibendum ut tristique et egestas quis ipsum \\\n",
        " suspendisse ultrices gravida dictum fusce ut placerat orci nulla pellentesque \\\n",
        " dignissim enim sit amet venenatis urna cursus eget nunc scelerisque viverra mauris \\\n",
        " in aliquam sem fringilla ut morbi tincidunt augue interdum velit euismod in \\\n",
        " pellentesque massa placerat duis ultricies lacus sed turpis tincidunt id aliquet \\\n",
        " risus feugiat in ante metus dictum at tempor commodo ullamcorper a lacus vestibulum \\\n",
        " sed arcu non odio euismod lacinia at quis risus sed vulputate odio ut enim blandit \\\n",
        " volutpat maecenas volutpat blandit aliquam etiam erat velit scelerisque in \\\n",
        " dictum non consectetur a erat nam at lectus urna\"\"\"\n",
        "\n",
        "print(\"The length in words is\", len(placeholder.split(' ')))\n",
        "print(\"The length in characters is\", len(placeholder))"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 500\n",
            "The length in characters is 3030\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5yG3-YncJfTB"
      },
      "source": [
        "# Permissions to use your summaries\n",
        "\n",
        "We'd like to use your summaries to help develop better educational resources like [remember.school](https://remember.school) or a question-answering system about the textbook.  \n",
        "\n",
        "1. Please enter your name.  \n",
        "\n",
        "2. If you grant permission for us to use your summaries under a [Creative Commons License](https://creativecommons.org/licenses/by/4.0/), please set ```permission = True```.  If you do not grant permission, please set it to false. Your answer won't affect your grade.\n",
        "\n",
        "3. If you grant us permission, and you'd like to remain anonymous, set ```anonymous = True```.  We'll remove your name before sharing your summaries with anyone. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2RECnJArJew4"
      },
      "source": [
        "name = \"\"\n",
        "permission = True\n",
        "anonymous = True"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GZZXyrV-xuAp"
      },
      "source": [
        "# Module 1 - Intro to AI"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y2NfE1CKxzS-"
      },
      "source": [
        "# Russell and Norvig, AIMA 4th Edition, Chapter 1 “Introduction”, Section 1.1 “What is AI”\n",
        "\n",
        "chapter1_1 = \"\"\" Different perspectives on AI: thinking vs. behavior; model humans vs. try to achieve the optimal results. \\\n",
        "AI is concerned with rational action. An intelligent agent takes the best possible action given what it knows. \"\"\" \n",
        "\n",
        "# Russell and Norvig, AIMA 4th Edition, Chapter 27 “Philosophical Foundations”, Sections 27.1 “The Limits of AI” and 27.2 “Can Machines Really Think?”\n",
        "chapter27_1 = \"\"\" Weak AI: machines could behave intelligently. Strong AI: machines have actual minds (not simulated minds) \"\"\"\n",
        "chapter27_2 = \"\"\" Question of conciousness is difficult but we focus on creating programs that behave intelligently. \"\"\"\n",
        "\n",
        "# Russell and Norvig, AIMA 4th Edition, Chapter 2 “Intelligent Agents” (whole chapter)\n",
        "chapter2_1 = \"\"\" Generally an agent's choice of action at any given time can depend on its built-in knowledge and on the the entire percept sequence observed so far,\\\n",
        "but not on the future (anything hasn't perceived).  \"\"\"\n",
        "chapter2_2 = \"\"\" At any give time what is rational depends on 4 elements. 1. performance measure. 2. agent's prior knowledge of environment. 3. actions that can be performed.\\\n",
        "4. agent's percept sequence up to the given time. Ominiscience is not required to be rational. \"\"\"\n",
        "chapter2_3 = \"\"\" PEAS: performance; environment; actuators; sensors.  known vs. unknown: not strictly a property of the environment but rather the agent's knowledge about the \\\n",
        "laws of physics of the environment. \"\"\"\n",
        "chapter2_4 = \"\"\" agent = architecture + program. Agent program implements agent function (mapping from percepts to actions). Simple reflex agents; model-based reflex agents\\\n",
        "(keeps track of the current state of the world using an internal model including transition and sensor models); goal-based agents; utility-based agents (best expected utility)\\\n",
        "Design of the agent programs depends on the nature of the environment.  \"\"\"\n",
        "\n",
        "# Lecture notes \n",
        "module1_lecture_notes = \"\"\" Think humanly, act humanly, think rationally, act rationally. We focus on act rationally.\n",
        "An agent is anything that perceives its environment through sensors and can act on its environment through actuators. \n",
        "A percept is the agent’s perceptual inputs at any given time. An agent is specified by an agent function f that maps a sequence of percept vectors P to an action a from a set A.\n",
        "The agent program runs on the physical architecture to produce f. agent = program + architecture. Agents include humans, robots, softbots, thermostats. \n",
        "Performance measure: an objective criterion for success of an agent's behavior; Rational agent: for each possible percept sequence P, a rational agent selects an action a \n",
        "that maximizes the expected value of its performance measure given the evidence provided by the percept sequence.\n",
        "Task environment: PEAS. Fully vs. partially observable; deterministic vs. nondeterministic vs. stochastic; episodic (atomic episodes) vs. sequential; static vs. dynamic; discrete vs. continuous; \n",
        "single vs. multi agent; known vs. unknown. Hardest: continuous; partially observable; stochastic; multiagent; unknown outcomes. \n",
        "When something behavior can best be described as having its own performance measure, then we should consider it to be an agent. \n",
        "A simple reflex agent is one that selects an action based on the current percept, and ignores the rest of the percept history. A problem-solving agent must plan ahead. \n",
        "It will consider a sequence of actions that form a path to a goal state. The computational process that it undertakes is called search \"\"\"\n",
        "\n",
        "\n",
        "# Definitions of concepts from Module 1.\n",
        "# You can define any bolded term in the textbook, \n",
        "# but you most use your own words for the definitions.\n",
        "# Each definition should be at most 20 words long.\n",
        "module1_definitions = {\n",
        "\"agent\" : \"anything that can perceive and act in an environment\",\n",
        "\"percept\" : \"agent's perceptual inputs at any time\",\n",
        "\"agent function\" : \"maps a sequence of percept vectors P to an action a from a set A\",\n",
        "\"agent program\": \"runs on the physical architecture to produce agent function\",\n",
        "}\n",
        "\n"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "elCCgY64spuq"
      },
      "source": [
        "## Preview of your notes for Module 1\n",
        "\n",
        "Here's a preview of your how your notes will be displayed.  At the end of this Python notebook, we'll aggregate your notes for all modules together into a single display.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gYYeNcdM_maE",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 971
        },
        "cellView": "form",
        "outputId": "6b391676-d1c6-48ed-9b95-4d91af763514"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "from IPython.display import display, Markdown, Latex\n",
        "\n",
        "module1_sections = [\n",
        "  (\"AIMA Chapter 1.1 What is AI?\", chapter1_1),\n",
        "  (\"AIMA Chapter 27.1 The Limits of AI\", chapter27_1),\n",
        "  (\"AIMA Chapter 27.1 Can Machines Really Think?\", chapter27_2),\n",
        "  (\"AIMA 2.1 Agents and Environments\", chapter2_1),\n",
        "  (\"AIMA 2.2 Good Behavior: The Concept of Rationality\", chapter2_2),\n",
        "  (\"AIMA 2.3 The Nature of Environments\", chapter2_3),\n",
        "  (\"AIMA 2.4 The Structure of Agents\", chapter2_4),\n",
        "  (\"Module 1 Lecture Notes\", module1_lecture_notes),\n",
        "]\n",
        "\n",
        "\n",
        "def length_check(sections):\n",
        "  warnings = \"\"\n",
        "  for section, summary in sections:\n",
        "    max_word_length = 500\n",
        "    max_character_length = 3000\n",
        "    if \"lecture\" in section.lower():\n",
        "      # It's OK to do longer summaries of the lectures\n",
        "      max_word_length = 2000\n",
        "      max_character_length = 12000  \n",
        "    if (len(summary) > max_character_length or len(summary.split(' ')) > max_word_length):\n",
        "      warnings += \"WARNING: Your summary of '{section}' is too long!\\n\".format(section=section)\n",
        "  return warnings\n",
        "\n",
        "def length_check_definitions(definitions):\n",
        "  warnings = \"\"\n",
        "  for concept, definition in definitions.items():\n",
        "    if(len(definition.split(' ')) > 20):\n",
        "      warnings += \"WARNING: Your definition of '{concept}' is too long!\\n\".format(concept=concept)\n",
        "  return warnings\n",
        "\n",
        "\n",
        "def format_module(module_num, title, sections, definitions, show_placeholder=False):\n",
        "  markdown = \"\"\n",
        "\n",
        "  # Check summary lengths\n",
        "  warnings = length_check(sections)\n",
        "  if warnings != \"\":\n",
        "    markdown += \"## \" + warnings\n",
        "\n",
        "  # Check definition lengths\n",
        "  warnings = length_check_definitions(definitions)\n",
        "  if warnings != \"\":\n",
        "    markdown += \"## \" + warnings\n",
        "\n",
        "  markdown += \"# Module {num}: {title}\\n\".format(num=module_num, title=title)\n",
        "\n",
        "\n",
        "  for section, summary in sections:\n",
        "    if show_placeholder and summary == \" Your summary here \":\n",
        "      summary = placeholder\n",
        "    markdown += \"## {section}\\n{summary}\\n\\n\".format(section=section, summary=summary)\n",
        "  \n",
        "  markdown += \"## Module {num} Definitions\\n\".format(num=module_num)\n",
        "  for concept, definition in definitions.items():\n",
        "    markdown += \"* **{concept}** - {definition}\\n\".format(concept=concept.capitalize(), definition=definition)\n",
        "  return markdown\n",
        "\n",
        "\n",
        "module1_markdown = format_module(1, \"Rational Agents and Task Environments\", module1_sections, module1_definitions)\n",
        "display(Markdown(module1_markdown))"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 1: Rational Agents and Task Environments\n## AIMA Chapter 1.1 What is AI?\n Different perspectives on AI: thinking vs. behavior; model humans vs. try to achieve the optimal results. AI is concerned with rational action. An intelligent agent takes the best possible action given what it knows. \n\n## AIMA Chapter 27.1 The Limits of AI\n Weak AI: machines could behave intelligently. Strong AI: machines have actual minds (not simulated minds) \n\n## AIMA Chapter 27.1 Can Machines Really Think?\n Question of conciousness is difficult but we focus on creating programs that behave intelligently. \n\n## AIMA 2.1 Agents and Environments\n Generally an agent's choice of action at any given time can depend on its built-in knowledge and on the the entire percept sequence observed so far,but not on the future (anything hasn't perceived).  \n\n## AIMA 2.2 Good Behavior: The Concept of Rationality\n At any give time what is rational depends on 4 elements. 1. performance measure. 2. agent's prior knowledge of environment. 3. actions that can be performed.4. agent's percept sequence up to the given time. Ominiscience is not required to be rational. \n\n## AIMA 2.3 The Nature of Environments\n PEAS: performance; environment; actuators; sensors.  known vs. unknown: not strictly a property of the environment but rather the agent's knowledge about the laws of physics of the environment. \n\n## AIMA 2.4 The Structure of Agents\n agent = architecture + program. Agent program implements agent function (mapping from percepts to actions). Simple reflex agents; model-based reflex agents(keeps track of the current state of the world using an internal model including transition and sensor models); goal-based agents; utility-based agents (best expected utility)Design of the agent programs depends on the nature of the environment.  \n\n## Module 1 Lecture Notes\n Think humanly, act humanly, think rationally, act rationally. We focus on act rationally.\nAn agent is anything that perceives its environment through sensors and can act on its environment through actuators. \nA percept is the agent’s perceptual inputs at any given time. An agent is specified by an agent function f that maps a sequence of percept vectors P to an action a from a set A.\nThe agent program runs on the physical architecture to produce f. agent = program + architecture. Agents include humans, robots, softbots, thermostats. \nPerformance measure: an objective criterion for success of an agent's behavior; Rational agent: for each possible percept sequence P, a rational agent selects an action a \nthat maximizes the expected value of its performance measure given the evidence provided by the percept sequence.\nTask environment: PEAS. Fully vs. partially observable; deterministic vs. nondeterministic vs. stochastic; episodic (atomic episodes) vs. sequential; static vs. dynamic; discrete vs. continuous; \nsingle vs. multi agent; known vs. unknown. Hardest: continuous; partially observable; stochastic; multiagent; unknown outcomes. \nWhen something behavior can best be described as having its own performance measure, then we should consider it to be an agent. \nA simple reflex agent is one that selects an action based on the current percept, and ignores the rest of the percept history. A problem-solving agent must plan ahead. \nIt will consider a sequence of actions that form a path to a goal state. The computational process that it undertakes is called search \n\n## Module 1 Definitions\n* **Agent** - anything that can perceive and act in an environment\n* **Percept** - agent's perceptual inputs at any time\n* **Agent function** - maps a sequence of percept vectors P to an action a from a set A\n* **Agent program** - runs on the physical architecture to produce agent function\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9xRbdqDox2zD"
      },
      "source": [
        "#Module 2 - Python"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DpBvu7EMx7sy"
      },
      "source": [
        "# No readings for this module.  Instead you can summarize the lecture instead.\n",
        "\n",
        "# Lecture notes \n",
        "module2_lecture_notes = \"\"\" \"\"\"\n",
        "\n",
        "# Each definition should be at most 20 words long.\n",
        "module2_definitions = {\n",
        "\"mutability\" : \"objects can be changed/muted after created.\",\n",
        "\"*args\": \"used to accept a variable number of non-keyword arguments to function.\",\n",
        "\"**kargs\": \"used to accept a variable number of keyword arguments to function\",\n",
        "\"dynamic typing\": \"variables come into existence when first assigned\",\n",
        "\"Generators\": \"one kind of iterators \",\n",
        "}"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ydOK__lwWgdp",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 252
        },
        "outputId": "849beb17-28d7-4b9f-c509-2170ff870372"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "\n",
        "module2_sections = [\n",
        "  (\"Module 2 Lecture Notes\", module2_lecture_notes),\n",
        "]\n",
        "\n",
        "module2_markdown = format_module(2, \"Python\", module2_sections, module2_definitions)\n",
        "display(Markdown(module2_markdown))"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 2: Python\n## Module 2 Lecture Notes\n \n\n## Module 2 Definitions\n* **Mutability** - objects can be changed/muted after created.\n* ***args** - used to accept a variable number of non-keyword arguments to function.\n* ****kargs** - used to accept a variable number of keyword arguments to function\n* **Dynamic typing** - variables come into existence when first assigned\n* **Generators** - one kind of iterators \n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "46MQLR-Gx8dZ"
      },
      "source": [
        "#Module 3 - Search Problems"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BN0b1eVYx-nd"
      },
      "source": [
        "# Russell and Norvig, AIMA 4th Edition, AIMA Chapter 3 “Problem Solving by Search” (3.1-3.4)\n",
        "chapter3_1 = \"\"\" Problem-solving agents plan ahead: consider a sequence of actions that form a path to the goal. A search problem: a set of possible states (state space);\\\n",
        "initial state; goal states; actions; transition model; action cost function.  \"\"\"\n",
        "chapter3_2 = \"\"\" route-finding problem; Touring problem (TSP); VLSI layout; robot navigation; automatic assembly sequencing \"\"\"\n",
        "chapter3_3 = \"\"\" Search tree: node represents a state in the state space and edges represent actions. Graph search checks for redundant paths and tree-like search doesn't. \\\n",
        "Three kinds of sequence are used in search algorithms. PriorityQueue used in best first search; FIFO used in BFS and LIFO(stack) used in DFS. Assess algorithm by completeness; \\\n",
        "cost optimality; time complexity and space complexity. Depth d: number of actions in an optimal solution; m: the max number of actions in any path; b: branching factor: number of \\\n",
        "successors of a node.\"\"\"\n",
        "chapter3_4 = \"\"\"BFS: nodes generated: 1+b+b^2+...+b^d=O(b^d). All nodes are in memory. Both time and space complexity O(b^d) for BFS. Complete and optimal.\\\n",
        "Dijkstra's algorithm (Uniform-cost search): actions have different costs. DFS (tree-like search): efficient and complete in finite state spaces. \\\n",
        "May get stuck in an infinite loop in cyclic state spaces. much smaller needs for space. IDS: fix shortcoming of DFS. Bidirectional search: simultaneously search forward \\\n",
        "from the initial state and backwards from the goal state, hoping the two searches will meet.   \"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module3_lecture_notes = \"\"\" Problem solving agent - goal; search problem (states, actions, performance measure) and solution. A problem is formulated by States, \\\n",
        "initial state, actions, transition model, path cost (performance measure), goal test. A solution is optimal if no solution has a lower path cost. \\\n",
        "State space: the set of all states reachable from the initial state by any sequence of actions. Frontier: those states that are available for expanding (for applying legal actions to). \\\n",
        "Tree search: treats different paths to the same node as distinct. A node is a data structure constituting part of a search tree. Tree search problem: repeated states. \\\n",
        "Graph search: check to see if a node has been visited before adding to search queue. must keep track of all possible states (can use a lot of memory). \\\n",
        "Uninformed search: all non-goal nodes in frontier looks equally good. Strategy: order of tree expansion (LIFO, FIFO, PriorityQ). \\\n",
        "b: max branching factor, d: depth of the shallowest goal node; m: max length of any path in the state space. BFS: explore shallowest node first. frontier is FIFO. \\\n",
        "node only inserted into queue after testing to see if it is a goal state. DFS: expand deepest unexpanded node, frontier is LIFO. \\\n",
        "Use DFS if space limited, many possible solutions with long paths; use BFS if possible infinite paths, some solutions have short paths\n",
        " \"\"\"\n",
        "\n",
        "# Definitions of concepts from Module 3.\n",
        "# You can define any bolded term in the textbook, \n",
        "# but you most use your own words for the definitions.\n",
        "# Each definition should be at most 20 words long.\n",
        "module3_definitions = {\n",
        "\"problem-solving agent\" : \"an agent that can formulate a plan via search\",    \n",
        "}\n",
        "\n"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lvtN3p7ET3Tx",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 723
        },
        "outputId": "3b2be44b-8cfd-4936-c12a-28ff0ccdb376"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module3_sections = [\n",
        "  (\"AIMA Chapter 3.1 Problem-Solving Agents\", chapter3_1),\n",
        "  (\"AIMA Chapter 3.2 Example Problems\", chapter3_2),\n",
        "  (\"AIMA Chapter 3.3 Search Algorithms\", chapter3_3),\n",
        "  (\"AIMA Chapter 3.4 Uninformed Search Strategies\", chapter3_4),\n",
        "  (\"Module 3 Lecture Notes\", module3_lecture_notes),\n",
        "]\n",
        "\n",
        "module3_markdown = format_module(3, \"Search Problems\", module3_sections, module3_definitions)\n",
        "display(Markdown(module3_markdown))"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 3: Search Problems\n## AIMA Chapter 3.1 Problem-Solving Agents\n Problem-solving agents plan ahead: consider a sequence of actions that form a path to the goal. A search problem: a set of possible states (state space);initial state; goal states; actions; transition model; action cost function.  \n\n## AIMA Chapter 3.2 Example Problems\n route-finding problem; Touring problem (TSP); VLSI layout; robot navigation; automatic assembly sequencing \n\n## AIMA Chapter 3.3 Search Algorithms\n Search tree: node represents a state in the state space and edges represent actions. Graph search checks for redundant paths and tree-like search doesn't. Three kinds of sequence are used in search algorithms. PriorityQueue used in best first search; FIFO used in BFS and LIFO(stack) used in DFS. Assess algorithm by completeness; cost optimality; time complexity and space complexity. Depth d: number of actions in an optimal solution; m: the max number of actions in any path; b: branching factor: number of successors of a node.\n\n## AIMA Chapter 3.4 Uninformed Search Strategies\nBFS: nodes generated: 1+b+b^2+...+b^d=O(b^d). All nodes are in memory. Both time and space complexity O(b^d) for BFS. Complete and optimal.Dijkstra's algorithm (Uniform-cost search): actions have different costs. DFS (tree-like search): efficient and complete in finite state spaces. May get stuck in an infinite loop in cyclic state spaces. much smaller needs for space. IDS: fix shortcoming of DFS. Bidirectional search: simultaneously search forward from the initial state and backwards from the goal state, hoping the two searches will meet.   \n\n## Module 3 Lecture Notes\n Problem solving agent - goal; search problem (states, actions, performance measure) and solution. A problem is formulated by States, initial state, actions, transition model, path cost (performance measure), goal test. A solution is optimal if no solution has a lower path cost. State space: the set of all states reachable from the initial state by any sequence of actions. Frontier: those states that are available for expanding (for applying legal actions to). Tree search: treats different paths to the same node as distinct. A node is a data structure constituting part of a search tree. Tree search problem: repeated states. Graph search: check to see if a node has been visited before adding to search queue. must keep track of all possible states (can use a lot of memory). Uninformed search: all non-goal nodes in frontier looks equally good. Strategy: order of tree expansion (LIFO, FIFO, PriorityQ). b: max branching factor, d: depth of the shallowest goal node; m: max length of any path in the state space. BFS: explore shallowest node first. frontier is FIFO. node only inserted into queue after testing to see if it is a goal state. DFS: expand deepest unexpanded node, frontier is LIFO. Use DFS if space limited, many possible solutions with long paths; use BFS if possible infinite paths, some solutions have short paths\n \n\n## Module 3 Definitions\n* **Problem-solving agent** - an agent that can formulate a plan via search\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WHREsHSSyA-h"
      },
      "source": [
        "#Module 4 - Informed Search"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tuFC8w7CyCns"
      },
      "source": [
        "# Russell and Norvig, AIMA Chapter 3 “Problem Solving by Search” (3.5-3.6)\n",
        "chapter3_5 = \"\"\" heuristic function h(n)=estimated cost of the cheapest path from the state at node n to a goal state. Greedy best-first search: f(n)=h(n). Greedy is completed \\\n",
        "in finite state spaces but not in infinite ones. A* search: f(n)=g(n)+h(n). g(n) is the path cost from the initial state to node n. f(n) is estimated cost of the best path that \\\n",
        "continues from n to goal state. Admissible heuristic: never overestimate the cost to reach a goal (optimistic). A heuristic is consistent if, for every node n and every successor n' \\\n",
        "of n generated by an action a, we have h(n)<=c(n,a,n')+h(n'). triangle inequality. Consistency implies admissibility. With a consistent heuristic A* is optimal. The first time a state \\\n",
        "is reached it will be on an optimal path, so never need to re-add a state to the frontier and never have to change an entry in reached set. Space complexity of A* is an issue. \\\n",
        "Iterative-deepening A*: cutoff is the f-cost (g+h). w/o requirement to keep all reached states in memory at a cost of revisiting some states multiple times. \\\n",
        "Recursive best-first search (RBFS): optimal if h(n) is admissible, space complexity is linear in the depth of the deepest optimal solution. IDA* and RBFS use too little memory. \\\n",
        "MA* and SMA*: SMA* works like A* expanding the best leaf until memory is full. Then dropping worst leaf node with the highest f-value. Optimal. \n",
        "Weighted A* search: weigh h(n) more heavily expand fewer nodes but sacrificing optimality. Beam search: limit size of the frontier keeping only the k nodes with the best scores. \\\n",
        "\"\"\"\n",
        "chapter3_6 = \"\"\" Effective branching factor b*. N+1=1+b*+(b*)^2+...+(b*)^d. If h2(n) and h1(n) are both admissible and h2(n)>=h1(n) for all n then h2 dominates h1. \\\n",
        "A* using h2 will not expand more nodes than A* using h1. Domination tranlates to efficiency. We can generate heuristic from relaxed problems. \"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module4_lecture_notes = \"\"\" Uniform cost search: misnomer. g(n): the path cost function. \n",
        "UCS: extension of BFS. Expand node with lowest path cost, frontier is PriorityQueue ordered by g(n). \n",
        "Important: tests if a node is a goal state when it is selected for expansion, not when it is added to the frontier. \n",
        "Better idea than UCS: node expansion based on an estimate which includes distance to the goal f(n). Sort frontier queue by this new f(n). \n",
        "Greedy; A* search. Greedy Best-First search: select node for expansion with minimal evaluation function f(n)=h(n). Exactly UCS with f(n) replacing g(n). \n",
        "Completely ignores g(n): the cost to get to n. Not optimal. Not complete – can stuck in loops. A* search: Best-known form of best-first search. Expand most promising first. \n",
        "f(n)=g(n) + h(n). A heuristic h(n) is admissible if it never overestimates the cost to reach the goal (optimistic). Theorem: If h(n) is admissible, A* using Tree Search is optimal.\n",
        "The cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem. If h2(n) and h1(n) are both admissible and h2(n)>=h1(n) then h2 dominates h1.\n",
        "UCS expands equally in all directions. A* expands mainly toward the goal, but does hedge its bets to ensure optimality. The space complexity of A* is O(b^d).  \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module4_definitions = {\n",
        "\"concept\" : \"definition\",    \n",
        "}\n",
        "\n"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vqtEs0gHs9sO",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 625
        },
        "outputId": "e5cd63ba-6148-4833-8eff-415af9dc2d39"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module4_sections = [\n",
        "  (\"AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\", chapter3_5),\n",
        "  (\"AIMA Chapter 3.6 Heuristic Functions\", chapter3_6),\n",
        "  (\"Module 4 Lecture Notes\", module4_lecture_notes),\n",
        "]\n",
        "\n",
        "module4_markdown = format_module(4, \"Informed Search\", \n",
        "                         module4_sections, \n",
        "                         module4_definitions)\n",
        "display(Markdown(module4_markdown))"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 4: Informed Search\n## AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\n heuristic function h(n)=estimated cost of the cheapest path from the state at node n to a goal state. Greedy best-first search: f(n)=h(n). Greedy is completed in finite state spaces but not in infinite ones. A* search: f(n)=g(n)+h(n). g(n) is the path cost from the initial state to node n. f(n) is estimated cost of the best path that continues from n to goal state. Admissible heuristic: never overestimate the cost to reach a goal (optimistic). A heuristic is consistent if, for every node n and every successor n' of n generated by an action a, we have h(n)<=c(n,a,n')+h(n'). triangle inequality. Consistency implies admissibility. With a consistent heuristic A* is optimal. The first time a state is reached it will be on an optimal path, so never need to re-add a state to the frontier and never have to change an entry in reached set. Space complexity of A* is an issue. Iterative-deepening A*: cutoff is the f-cost (g+h). w/o requirement to keep all reached states in memory at a cost of revisiting some states multiple times. Recursive best-first search (RBFS): optimal if h(n) is admissible, space complexity is linear in the depth of the deepest optimal solution. IDA* and RBFS use too little memory. MA* and SMA*: SMA* works like A* expanding the best leaf until memory is full. Then dropping worst leaf node with the highest f-value. Optimal. \nWeighted A* search: weigh h(n) more heavily expand fewer nodes but sacrificing optimality. Beam search: limit size of the frontier keeping only the k nodes with the best scores. \n\n## AIMA Chapter 3.6 Heuristic Functions\n Effective branching factor b*. N+1=1+b*+(b*)^2+...+(b*)^d. If h2(n) and h1(n) are both admissible and h2(n)>=h1(n) for all n then h2 dominates h1. A* using h2 will not expand more nodes than A* using h1. Domination tranlates to efficiency. We can generate heuristic from relaxed problems. \n\n## Module 4 Lecture Notes\n Uniform cost search: misnomer. g(n): the path cost function. \nUCS: extension of BFS. Expand node with lowest path cost, frontier is PriorityQueue ordered by g(n). \nImportant: tests if a node is a goal state when it is selected for expansion, not when it is added to the frontier. \nBetter idea than UCS: node expansion based on an estimate which includes distance to the goal f(n). Sort frontier queue by this new f(n). \nGreedy; A* search. Greedy Best-First search: select node for expansion with minimal evaluation function f(n)=h(n). Exactly UCS with f(n) replacing g(n). \nCompletely ignores g(n): the cost to get to n. Not optimal. Not complete – can stuck in loops. A* search: Best-known form of best-first search. Expand most promising first. \nf(n)=g(n) + h(n). A heuristic h(n) is admissible if it never overestimates the cost to reach the goal (optimistic). Theorem: If h(n) is admissible, A* using Tree Search is optimal.\nThe cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem. If h2(n) and h1(n) are both admissible and h2(n)>=h1(n) then h2 dominates h1.\nUCS expands equally in all directions. A* expands mainly toward the goal, but does hedge its bets to ensure optimality. The space complexity of A* is O(b^d).  \n\n## Module 4 Definitions\n* **Concept** - definition\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F3FPMzG9yUls"
      },
      "source": [
        "#Module 5 - Adversarial Search and Games"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iEIOpL7vyWcc"
      },
      "source": [
        "# Russell and Norvig, AIMA chapter 5 \"Adversarial Search\"”\" (5.1-5.3, and 5.5).\n",
        "chapter5_1 = \"\"\"  \"\"\"\n",
        "chapter5_2 = \"\"\"  \"\"\"\n",
        "chapter5_3 = \"\"\" Heuristic alpha-beta tree search. Due to limited computation time we can cut off search early and apply a heuristic evaluation functions to states, \\\n",
        "effectively treating nonterminal nodes as if they were terminal. Instead of using UTILITY function we use EVAL which estimates a state's utility. EVAL(state, player) \\\n",
        "estimates the expected utility of state to player. For terminal states, eval(s,p)=utility(s,p). \"\"\"\n",
        "chapter5_5 = \"\"\" Stochastic games: expectiminimax value. Chance nodes in addition to MAX and MIN nodes.  \"\"\"\n",
        "\n",
        "# Russell and Norvig, AIMA Chapter 16 “Making Simple Decisions” (16.1-16.3)\n",
        "chapter16_1 = \"\"\" agent's preference captured by utility function U(s). Expected utility of an action given the evidence EU(a). Maximum expected utility.  \"\"\"\n",
        "chapter16_2 = \"\"\" Orderability: for any two lotteries an agent must either prefer one or indifferent. Transitivity: A>B & B>C then A>C. Continuity: A>B>C then exists p \\\n",
        "such that pA+(1-p)B=C. Substitutability; Monotonicity: if agent prefers A to B the the agent must prefer the lottery with higher prob for A; Decomposability: compound lotteries \\\n",
        "can be decomped into simpler ones. \"\"\"\n",
        "chapter16_3 = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Lecture notes \n",
        "module5_lecture_notes = \"\"\" Simplest game environment: multiagent, static, discrete, fully observable, strategic. \n",
        "Games as search: MAX & MIN. Initial state, successor function(list of (move,state) pairs specifying legal moves), terminal test, utility function. \n",
        "MAX uses search tree to find next move. Game trees: nodes represent board positions and edges are legal moves. All play is computed from MAX’s vantage point. \n",
        "Even levels (MAX) & odd levels (MIN). Evaluation function f(n): evaluates how good a ‘board position’ is based on static features of that board alone. \n",
        "e.g. tic-tac-toe: f(n)=#of 3 lengths open for MAX-#open for MIN. In non-terminal state, evaluation function gives an estimate of the expected utility of the game from a given position. \n",
        "Minimax Rule: recursive process. For a MAX node, the backed-up value is the maximum of the values of its children. For a MIN node, the backed-up value is the minimum of the values of its children. \n",
        "Alpha-Beta pruning: alpha: MAX’s current lower bound on MAX’s outcome (at least). beta: MIN’s current upper bound for MIN’s outcome (at most). \n",
        "Stop evaluating a branch: when eval a MAX node, a value >= beta is backed-up; when eval a MIN node, a value <= alpha is backed-up. \n",
        "Update alpha & prune by beta in MAX node; update beta & prune by alpha in MIN node. The effectiveness of alpha-beta pruning depends on the ordering in which the successors \n",
        "are examined. A limitation of alpha-beta pruning is the assumption that the leaf evaluations are randomly distributed across last layer of the tree. \n",
        "Expectimax search: compute the average score under optimal play. Chance nodes are like min nodes but the outcome is uncertain.  \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module5_definitions = {\n",
        "\"concept\" : \"definition\",    \n",
        "}\n",
        "\n"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tFo_aRsvKNrZ"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "KUc_LgzDvWf-",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 858
        },
        "outputId": "5e437984-cb29-4dc6-deee-6f39ccc3d304"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module5_sections = [\n",
        "  (\"AIMA Chapter 5.1 Game Theory\", chapter5_1),\n",
        "  (\"AIMA Chapter 5.2 Optimal Decisions in Games\", chapter5_2),\n",
        "  (\"AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\", chapter5_3),\n",
        "  (\"AIMA Chapter 5.5 Stochastic Games\", chapter5_5),\n",
        "  (\"AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\", chapter16_1),\n",
        "  (\"AIMA Chapter 16.2 The Basis of Utility Theory\", chapter16_2),\n",
        "  (\"AIMA Chapter 16.3 Utility Functions\", chapter16_3),\n",
        "  (\"Module 5 Lecture Notes\", module5_lecture_notes),\n",
        "]\n",
        "\n",
        "module5_markdown = format_module(5, \"Adversarial Search and Games\", \n",
        "                         module5_sections, \n",
        "                         module5_definitions)\n",
        "display(Markdown(module5_markdown))"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 5: Adversarial Search and Games\n## AIMA Chapter 5.1 Game Theory\n  \n\n## AIMA Chapter 5.2 Optimal Decisions in Games\n  \n\n## AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\n Heuristic alpha-beta tree search. Due to limited computation time we can cut off search early and apply a heuristic evaluation functions to states, effectively treating nonterminal nodes as if they were terminal. Instead of using UTILITY function we use EVAL which estimates a state's utility. EVAL(state, player) estimates the expected utility of state to player. For terminal states, eval(s,p)=utility(s,p). \n\n## AIMA Chapter 5.5 Stochastic Games\n Stochastic games: expectiminimax value. Chance nodes in addition to MAX and MIN nodes.  \n\n## AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\n agent's preference captured by utility function U(s). Expected utility of an action given the evidence EU(a). Maximum expected utility.  \n\n## AIMA Chapter 16.2 The Basis of Utility Theory\n Orderability: for any two lotteries an agent must either prefer one or indifferent. Transitivity: A>B & B>C then A>C. Continuity: A>B>C then exists p such that pA+(1-p)B=C. Substitutability; Monotonicity: if agent prefers A to B the the agent must prefer the lottery with higher prob for A; Decomposability: compound lotteries can be decomped into simpler ones. \n\n## AIMA Chapter 16.3 Utility Functions\n Your summary here \n\n## Module 5 Lecture Notes\n Simplest game environment: multiagent, static, discrete, fully observable, strategic. \nGames as search: MAX & MIN. Initial state, successor function(list of (move,state) pairs specifying legal moves), terminal test, utility function. \nMAX uses search tree to find next move. Game trees: nodes represent board positions and edges are legal moves. All play is computed from MAX’s vantage point. \nEven levels (MAX) & odd levels (MIN). Evaluation function f(n): evaluates how good a ‘board position’ is based on static features of that board alone. \ne.g. tic-tac-toe: f(n)=#of 3 lengths open for MAX-#open for MIN. In non-terminal state, evaluation function gives an estimate of the expected utility of the game from a given position. \nMinimax Rule: recursive process. For a MAX node, the backed-up value is the maximum of the values of its children. For a MIN node, the backed-up value is the minimum of the values of its children. \nAlpha-Beta pruning: alpha: MAX’s current lower bound on MAX’s outcome (at least). beta: MIN’s current upper bound for MIN’s outcome (at most). \nStop evaluating a branch: when eval a MAX node, a value >= beta is backed-up; when eval a MIN node, a value <= alpha is backed-up. \nUpdate alpha & prune by beta in MAX node; update beta & prune by alpha in MIN node. The effectiveness of alpha-beta pruning depends on the ordering in which the successors \nare examined. A limitation of alpha-beta pruning is the assumption that the leaf evaluations are randomly distributed across last layer of the tree. \nExpectimax search: compute the average score under optimal play. Chance nodes are like min nodes but the outcome is uncertain.  \n\n## Module 5 Definitions\n* **Concept** - definition\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qVSQhTVSyYYO"
      },
      "source": [
        "#Module 6 - \"Constraint Satisfaction Problems\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wm4mfW82yaRH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "85c4defb-c7a9-4069-b6fd-27d04ed49830"
      },
      "source": [
        "# Russell and Norvig, AIMA, Chapter 6 “Constraint Satisfaction Problems” (6.1-6.5)\n",
        "chapter6_1 = \"\"\" Your summary here \"\"\"\n",
        "chapter6_2 = \"\"\" Your summary here \"\"\"\n",
        "chapter6_3 = \"\"\" Your summary here \"\"\"\n",
        "chapter6_4 = \"\"\" Your summary here \"\"\"\n",
        "chapter6_5 = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module6_lecture_notes = \"\"\" CSP: specialized for identification problems. The goal itself is important, not the path. All paths at the same depth (for some formulations). Contrary to planning problems where the path to the goal is the important thing and paths have various costs, depths. A CSP consists of: Finite set of variables X1, X2, …., Xn, with nonempty domain of possible values for each variable D1, D2, …, Dn. Finite set of constraints C1, C2, …, Cm. A state is defined as an assignment of values to some or all variables. A consistent assignment does not violate the constraints. An assignment is complete when every variable is assigned a value. A solution to a CSP is a complete, consistent assignment. CSP representation: Constraint graph - nodes are variables and arcs are (binary) constraints. Types of CSPs: discrete variables. 1. finite domains O(d^n). 2. infinite domains: integers, strings. job scheduling, variables are start/end days for each job. Continuous variables: start/end times for Hubble Space Telescope observations. linear constraints solvable in polynomial time by linear programming. Types of constraints: unary, binary, higher-order (constraints involve 3 or more variables). preference (soft constraints). e.g. red is better than green. How to solve CSP? Idea 1 as a search problem. Solution is always found at depth n, for n variables (can use DFS).  n variables of domain size d. Branching factor at the root is n*d. Branching factor at next level is (n-1)*d. Tree has n!*d^n leaves. The variable assignments are commutative. e.g, [ step 1: WA = red; step 2: NT = green ] is equivalent to [ step 1: NT = green; step 2: WA = red ]. Therefore, a tree search, not a graph search. Only need to consider assignments to a single variable at each node. b = d and there are d^n leaves (n variables, domain size d ). DFS for CSPs with single-variable assignments is called backtracking search. basic uninformed algorithm for CSPs. Idea 2 improving backtrack efficiency. Heuristics: 1. Which variable should be assigned next? Most constrained variable & (if ties:) Most constraining variable. minimum remaining values (MRV) heuristic. 2. In what order should that variable’s values be tried? Least constraining value. 3. Use Forward checking. Keep track of remaining legal values for unassigned variables. Terminate search when any unassigned variable has no remaining legal values. Constraint propagation goes beyond forward checking & repeatedly enforces constraints locally. Idea 3 (big idea): Inference in CSPs. CSP solvers combine search and inference. Node consistency: satisfies unary constraints. Arc consistency: satisfies binary constraints. (Xi is arc-consistent w.r.t. Xj if for every value v in Di, there is some value w in Dj that satisfies the binary constraint on the arc between Xi and Xj). Edges to arcs: for a connected pair of nodes Xi and Xj , there are two arcs that connect them: (i,j) and (j,i). Given d values, checking arc (i,j) takes O(d^2) time worst case. Waltz/Mackworth towards AC-3. AC-3 complexity: worst case. all nodes can be connected to every other node. so total # of arcs is 2*n*(n-1), i.e. O(n^2). If there are d values, checking arc (i,j) takes O(d^2) time. Each arc (i,j) can only be inserted into the queue d times. Worst case complexity: O(n^2d^3). For planar constraint graphs, the number of arcs can only be linear in N and the time complexity is only O(nd^3). Beyond binary constraints: path consistency; global constraints. in Sudoko, all numbers in a row must be different. Backjumping: Improved backtracking. Fix using conflict-directed backjumping. Go back to predecessors of conflict set.\"\"\"\n",
        "\n",
        "# Definitions\n",
        "module6_definitions = {\n",
        "\"concept\" : \"definition\",    \n",
        "}\n",
        "\n",
        "print(\"The length in words is\", len(module6_lecture_notes.split(' ')))\n",
        "print(\"The length in characters is\", len(module6_lecture_notes))\n"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 591\n",
            "The length in characters is 3643\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "kKWbCsbfxL3R",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 906
        },
        "outputId": "e86281e2-25e8-460a-9bf1-6081e81921b4"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module6_sections = [\n",
        "  (\"AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\", chapter6_1),\n",
        "  (\"AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\", chapter6_2),\n",
        "  (\"AIMA Chapter 6.3 Backtracking Search for CSPs\", chapter6_3),\n",
        "  (\"AIMA Chapter 6.4 Local Search for CSPs\", chapter6_4),\n",
        "  (\"AIMA Chapter 6.5 The Structure of Problems\", chapter6_5),\n",
        "  (\"Module 6 Lecture Notes\", module6_lecture_notes),\n",
        "]\n",
        "\n",
        "module6_markdown = format_module(6, \"Constraint Satisfaction Problems\", \n",
        "                         module6_sections, \n",
        "                         module6_definitions)\n",
        "display(Markdown(module6_markdown))"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 6: Constraint Satisfaction Problems\n## AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\n Your summary here \n\n## AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\n Your summary here \n\n## AIMA Chapter 6.3 Backtracking Search for CSPs\n Your summary here \n\n## AIMA Chapter 6.4 Local Search for CSPs\n Your summary here \n\n## AIMA Chapter 6.5 The Structure of Problems\n Your summary here \n\n## Module 6 Lecture Notes\n CSP: specialized for identification problems. The goal itself is important, not the path. All paths at the same depth (for some formulations). Contrary to planning problems where the path to the goal is the important thing and paths have various costs, depths. A CSP consists of: Finite set of variables X1, X2, …., Xn, with nonempty domain of possible values for each variable D1, D2, …, Dn. Finite set of constraints C1, C2, …, Cm. A state is defined as an assignment of values to some or all variables. A consistent assignment does not violate the constraints. An assignment is complete when every variable is assigned a value. A solution to a CSP is a complete, consistent assignment. CSP representation: Constraint graph - nodes are variables and arcs are (binary) constraints. Types of CSPs: discrete variables. 1. finite domains O(d^n). 2. infinite domains: integers, strings. job scheduling, variables are start/end days for each job. Continuous variables: start/end times for Hubble Space Telescope observations. linear constraints solvable in polynomial time by linear programming. Types of constraints: unary, binary, higher-order (constraints involve 3 or more variables). preference (soft constraints). e.g. red is better than green. How to solve CSP? Idea 1 as a search problem. Solution is always found at depth n, for n variables (can use DFS).  n variables of domain size d. Branching factor at the root is n*d. Branching factor at next level is (n-1)*d. Tree has n!*d^n leaves. The variable assignments are commutative. e.g, [ step 1: WA = red; step 2: NT = green ] is equivalent to [ step 1: NT = green; step 2: WA = red ]. Therefore, a tree search, not a graph search. Only need to consider assignments to a single variable at each node. b = d and there are d^n leaves (n variables, domain size d ). DFS for CSPs with single-variable assignments is called backtracking search. basic uninformed algorithm for CSPs. Idea 2 improving backtrack efficiency. Heuristics: 1. Which variable should be assigned next? Most constrained variable & (if ties:) Most constraining variable. minimum remaining values (MRV) heuristic. 2. In what order should that variable’s values be tried? Least constraining value. 3. Use Forward checking. Keep track of remaining legal values for unassigned variables. Terminate search when any unassigned variable has no remaining legal values. Constraint propagation goes beyond forward checking & repeatedly enforces constraints locally. Idea 3 (big idea): Inference in CSPs. CSP solvers combine search and inference. Node consistency: satisfies unary constraints. Arc consistency: satisfies binary constraints. (Xi is arc-consistent w.r.t. Xj if for every value v in Di, there is some value w in Dj that satisfies the binary constraint on the arc between Xi and Xj). Edges to arcs: for a connected pair of nodes Xi and Xj , there are two arcs that connect them: (i,j) and (j,i). Given d values, checking arc (i,j) takes O(d^2) time worst case. Waltz/Mackworth towards AC-3. AC-3 complexity: worst case. all nodes can be connected to every other node. so total # of arcs is 2*n*(n-1), i.e. O(n^2). If there are d values, checking arc (i,j) takes O(d^2) time. Each arc (i,j) can only be inserted into the queue d times. Worst case complexity: O(n^2d^3). For planar constraint graphs, the number of arcs can only be linear in N and the time complexity is only O(nd^3). Beyond binary constraints: path consistency; global constraints. in Sudoko, all numbers in a row must be different. Backjumping: Improved backtracking. Fix using conflict-directed backjumping. Go back to predecessors of conflict set.\n\n## Module 6 Definitions\n* **Concept** - definition\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RYESi2kMybL8"
      },
      "source": [
        "#Module 7 - Logical Agents"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qIRJ4YYCzNHL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b53d7a15-7506-4b8f-f3e8-742f15e0d93c"
      },
      "source": [
        "# Russell and Norvig, AIMA Chapter 7 “Logical Agents” (Sections 7.1-7.5)\n",
        "\n",
        "chapter7_1 = \"\"\" Your summary here \"\"\"\n",
        "chapter7_2 = \"\"\" Your summary here \"\"\"\n",
        "chapter7_3 = \"\"\" Your summary here \"\"\"\n",
        "chapter7_4 = \"\"\" Your summary here \"\"\"\n",
        "chapter7_5 = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Lecture notes \n",
        "module7_lecture_notes = \"\"\" Logical agents: Knowledge-based agents use a process of reasoning over an internal representation of knowledge to decide what action to take. A KB contains a set of sentence that are written in a knowledge representation language. The sentence contains some assertion about the world. Axioms and derived sentences. The process of deriving new sentences from old sentences is called inference. In addition to asserting new knowledge into its KB, a knowledge-based agent can also query the KB and ask it to derive new knowledge in order to select what action it should take. Wampa world: Deterministic, discrete, static, single-agent, sequential and partially-observable. Models are mathematical abstractions that have a fixed set of truth values which are {true, false} for each sentence. If sentence a is true in model m then we say m satisfies a or m is a model of a. M(a) means the set of all models of a. Logical reasoning involves the entailment relation between sentence. entailment is the idea that a sentence follows logically from another sentence. alpha entails beta iff M(alpha) is subset of M(beta). alpha is more specific, or stronger than beta. Possible Worlds is the process of enumerating all Possible Worlds that are compatible with the KB. The consequences of a KB is a large set of additional sentences that are entailed given the sentences that have been added to the KB. Design inference algorithms to enumerate these sentences. Sentence alpha is derived from KB with algorithm i. Propositional logic: atomic sentences are represented with a single propositional symbol. Propositional symbols stand for a statement that can be true or false. e.g. W13 stands for there is a Wampa at location [1,3]. Soundness – the inference algorithm should only derive entailed sentences. Completeness – an inference algorithm is complete if it can derive all sentences that are entailed. Monotonicity: we can safely add to the KB, without invalidating anything else that we inferred. Inference rules: And Elimination; Bidirectional Elimination; Modus Ponens. There is a single inference rule, resolution, which yields a complete inference algorithm when coupled with any complete search algorithm. The resolution rule only applies to clauses (disjunction of literals). Every sentence of propositional logic is equivalent to a conjunction of clauses. CNF. Inference procedures based on resolution work using the principle of proof by contradiction. To show KB entails alpha we show {KB and not(alpha)} is not unsatisfiable. First, convert {KB and not(alpha)} into CNF. Then apply the resolution rule to the resulting clauses. Resolve every pair that contains complementary literals to produce a new clause. If that new clause isn’t already present, then add it to the set. Continue the process until one of two things happens: 1. There are no new clauses that can be added. In this case, the KB does not entail alpha. 2. Two clauses resolve to yield the empty clause, in which case the KB does entail alpha. The resolution algorithm is complete. Definite clause – disjunction of literals where exactly one is positive. Horn clause – disjunction of literals where at most one is positive. Horn clauses are closed under resolution. If you apply resolution to a horn clause, you get back a horn clause. Inference with Horn clauses can be done through the forward-chaining and backward-chaining algorithms. Deciding entailment with Horn clauses can be done in linear time with respect to the size of the KB. Forward Chaining is a kind of data-driven reasoning. An agent can start with known data, adding premises as new percepts come in, and then incrementally apply the forward chaining algorithm to derive new conclusions. It doesn’t have to have a specific query in mind to do so. Backward chaining is a form of goal-directed reasoning by working backwards from a query. If the query is known to be true, then no work is needed. Otherwise, the algorithm finds implications in the KB whose conclusions is the query. If all the premises of one of those can be proved to be true (via backward chaining), then the query is true. \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module7_definitions = {\n",
        "\"concept\" : \"definition\",    \n",
        "}\n",
        "\n",
        "print(\"The length in words is\", len(module7_lecture_notes.split(' ')))\n",
        "print(\"The length in characters is\", len(module7_lecture_notes))"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 674\n",
            "The length in characters is 4129\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "lXNcHuKjyNXa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 957
        },
        "outputId": "7d0ffe65-d466-4749-c000-e7d450a5a740"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module7_sections = [\n",
        "  (\"AIMA Chapter 7.1 Knowledge-Based Agents\", chapter7_1),\n",
        "  (\"AIMA Chapter 7.2 The Wumpus World\", chapter7_2),\n",
        "  (\"AIMA Chapter 7.3 Logic\", chapter7_3),\n",
        "  (\"AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\", chapter7_4),\n",
        "  (\"AIMA 7.5 Propositional Theorem Proving\", chapter7_5),\n",
        "  (\"Module 7 Lecture Notes\", module7_lecture_notes),\n",
        "]\n",
        "\n",
        "module7_markdown = format_module(7, \"Logical Agents\", \n",
        "                         module7_sections, \n",
        "                         module7_definitions)\n",
        "display(Markdown(module7_markdown))"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 7: Logical Agents\n## AIMA Chapter 7.1 Knowledge-Based Agents\n Your summary here \n\n## AIMA Chapter 7.2 The Wumpus World\n Your summary here \n\n## AIMA Chapter 7.3 Logic\n Your summary here \n\n## AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\n Your summary here \n\n## AIMA 7.5 Propositional Theorem Proving\n Your summary here \n\n## Module 7 Lecture Notes\n Logical agents: Knowledge-based agents use a process of reasoning over an internal representation of knowledge to decide what action to take. A KB contains a set of sentence that are written in a knowledge representation language. The sentence contains some assertion about the world. Axioms and derived sentences. The process of deriving new sentences from old sentences is called inference. In addition to asserting new knowledge into its KB, a knowledge-based agent can also query the KB and ask it to derive new knowledge in order to select what action it should take. Wampa world: Deterministic, discrete, static, single-agent, sequential and partially-observable. Models are mathematical abstractions that have a fixed set of truth values which are {true, false} for each sentence. If sentence a is true in model m then we say m satisfies a or m is a model of a. M(a) means the set of all models of a. Logical reasoning involves the entailment relation between sentence. entailment is the idea that a sentence follows logically from another sentence. alpha entails beta iff M(alpha) is subset of M(beta). alpha is more specific, or stronger than beta. Possible Worlds is the process of enumerating all Possible Worlds that are compatible with the KB. The consequences of a KB is a large set of additional sentences that are entailed given the sentences that have been added to the KB. Design inference algorithms to enumerate these sentences. Sentence alpha is derived from KB with algorithm i. Propositional logic: atomic sentences are represented with a single propositional symbol. Propositional symbols stand for a statement that can be true or false. e.g. W13 stands for there is a Wampa at location [1,3]. Soundness – the inference algorithm should only derive entailed sentences. Completeness – an inference algorithm is complete if it can derive all sentences that are entailed. Monotonicity: we can safely add to the KB, without invalidating anything else that we inferred. Inference rules: And Elimination; Bidirectional Elimination; Modus Ponens. There is a single inference rule, resolution, which yields a complete inference algorithm when coupled with any complete search algorithm. The resolution rule only applies to clauses (disjunction of literals). Every sentence of propositional logic is equivalent to a conjunction of clauses. CNF. Inference procedures based on resolution work using the principle of proof by contradiction. To show KB entails alpha we show {KB and not(alpha)} is not unsatisfiable. First, convert {KB and not(alpha)} into CNF. Then apply the resolution rule to the resulting clauses. Resolve every pair that contains complementary literals to produce a new clause. If that new clause isn’t already present, then add it to the set. Continue the process until one of two things happens: 1. There are no new clauses that can be added. In this case, the KB does not entail alpha. 2. Two clauses resolve to yield the empty clause, in which case the KB does entail alpha. The resolution algorithm is complete. Definite clause – disjunction of literals where exactly one is positive. Horn clause – disjunction of literals where at most one is positive. Horn clauses are closed under resolution. If you apply resolution to a horn clause, you get back a horn clause. Inference with Horn clauses can be done through the forward-chaining and backward-chaining algorithms. Deciding entailment with Horn clauses can be done in linear time with respect to the size of the KB. Forward Chaining is a kind of data-driven reasoning. An agent can start with known data, adding premises as new percepts come in, and then incrementally apply the forward chaining algorithm to derive new conclusions. It doesn’t have to have a specific query in mind to do so. Backward chaining is a form of goal-directed reasoning by working backwards from a query. If the query is known to be true, then no work is needed. Otherwise, the algorithm finds implications in the KB whose conclusions is the query. If all the premises of one of those can be proved to be true (via backward chaining), then the query is true. \n\n## Module 7 Definitions\n* **Concept** - definition\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vX7lbnJd4sT5"
      },
      "source": [
        "# Your Exam Notes\n",
        "\n",
        "Here are your full set of exam notes that you can use on your midterm.  Choose **Runtime > Run all**, then your notes will be displayed.\n",
        "\n",
        "If you are taking the exam in person, then you must print a copy of your notes for yourself.   Here's how to print:\n",
        "* Run the cell below. \n",
        "* Click its \"Mirror cell in tab\" button (this is located in the upper right corner of the cell, immediately to the right of the trash can).  This will open the cell.  \n",
        "* In Chrome, you can then print your notes using the **File > Print** option from the browser's menu.  \n",
        "* In the print options, change the layout from Landscape to Portrait. \n",
        "* Clicking the Save, will save a PDF that you can then print.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "T3cm1WV542_a",
        "outputId": "d7ebd676-743d-4489-8133-0df6f569206e"
      },
      "source": [
        "# This will generate the full set of notes that you can use for your exam.\n",
        "full_markdown = \"\\n\\n\".join([module1_markdown, \n",
        "                             module2_markdown, \n",
        "                             module3_markdown, \n",
        "                             module4_markdown, \n",
        "                             module5_markdown,\n",
        "                             module6_markdown,\n",
        "                             module7_markdown,]\n",
        "                       )\n",
        "display(Markdown(full_markdown))"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 1: Rational Agents and Task Environments\n## AIMA Chapter 1.1 What is AI?\n Different perspectives on AI: thinking vs. behavior; model humans vs. try to achieve the optimal results. AI is concerned with rational action. An intelligent agent takes the best possible action given what it knows. \n\n## AIMA Chapter 27.1 The Limits of AI\n Weak AI: machines could behave intelligently. Strong AI: machines have actual minds (not simulated minds) \n\n## AIMA Chapter 27.1 Can Machines Really Think?\n Question of conciousness is difficult but we focus on creating programs that behave intelligently. \n\n## AIMA 2.1 Agents and Environments\n Generally an agent's choice of action at any given time can depend on its built-in knowledge and on the the entire percept sequence observed so far,but not on the future (anything hasn't perceived).  \n\n## AIMA 2.2 Good Behavior: The Concept of Rationality\n At any give time what is rational depends on 4 elements. 1. performance measure. 2. agent's prior knowledge of environment. 3. actions that can be performed.4. agent's percept sequence up to the given time. Ominiscience is not required to be rational. \n\n## AIMA 2.3 The Nature of Environments\n PEAS: performance; environment; actuators; sensors.  known vs. unknown: not strictly a property of the environment but rather the agent's knowledge about the laws of physics of the environment. \n\n## AIMA 2.4 The Structure of Agents\n agent = architecture + program. Agent program implements agent function (mapping from percepts to actions). Simple reflex agents; model-based reflex agents(keeps track of the current state of the world using an internal model including transition and sensor models); goal-based agents; utility-based agents (best expected utility)Design of the agent programs depends on the nature of the environment.  \n\n## Module 1 Lecture Notes\n Think humanly, act humanly, think rationally, act rationally. We focus on act rationally.\nAn agent is anything that perceives its environment through sensors and can act on its environment through actuators. \nA percept is the agent’s perceptual inputs at any given time. An agent is specified by an agent function f that maps a sequence of percept vectors P to an action a from a set A.\nThe agent program runs on the physical architecture to produce f. agent = program + architecture. Agents include humans, robots, softbots, thermostats. \nPerformance measure: an objective criterion for success of an agent's behavior; Rational agent: for each possible percept sequence P, a rational agent selects an action a \nthat maximizes the expected value of its performance measure given the evidence provided by the percept sequence.\nTask environment: PEAS. Fully vs. partially observable; deterministic vs. nondeterministic vs. stochastic; episodic (atomic episodes) vs. sequential; static vs. dynamic; discrete vs. continuous; \nsingle vs. multi agent; known vs. unknown. Hardest: continuous; partially observable; stochastic; multiagent; unknown outcomes. \nWhen something behavior can best be described as having its own performance measure, then we should consider it to be an agent. \nA simple reflex agent is one that selects an action based on the current percept, and ignores the rest of the percept history. A problem-solving agent must plan ahead. \nIt will consider a sequence of actions that form a path to a goal state. The computational process that it undertakes is called search \n\n## Module 1 Definitions\n* **Agent** - anything that can perceive and act in an environment\n* **Percept** - agent's perceptual inputs at any time\n* **Agent function** - maps a sequence of percept vectors P to an action a from a set A\n* **Agent program** - runs on the physical architecture to produce agent function\n\n\n# Module 2: Python\n## Module 2 Lecture Notes\n \n\n## Module 2 Definitions\n* **Mutability** - objects can be changed/muted after created.\n* ***args** - used to accept a variable number of non-keyword arguments to function.\n* ****kargs** - used to accept a variable number of keyword arguments to function\n* **Dynamic typing** - variables come into existence when first assigned\n* **Generators** - one kind of iterators \n\n\n# Module 3: Search Problems\n## AIMA Chapter 3.1 Problem-Solving Agents\n Problem-solving agents plan ahead: consider a sequence of actions that form a path to the goal. A search problem: a set of possible states (state space);initial state; goal states; actions; transition model; action cost function.  \n\n## AIMA Chapter 3.2 Example Problems\n route-finding problem; Touring problem (TSP); VLSI layout; robot navigation; automatic assembly sequencing \n\n## AIMA Chapter 3.3 Search Algorithms\n Search tree: node represents a state in the state space and edges represent actions. Graph search checks for redundant paths and tree-like search doesn't. Three kinds of sequence are used in search algorithms. PriorityQueue used in best first search; FIFO used in BFS and LIFO(stack) used in DFS. Assess algorithm by completeness; cost optimality; time complexity and space complexity. Depth d: number of actions in an optimal solution; m: the max number of actions in any path; b: branching factor: number of successors of a node.\n\n## AIMA Chapter 3.4 Uninformed Search Strategies\nBFS: nodes generated: 1+b+b^2+...+b^d=O(b^d). All nodes are in memory. Both time and space complexity O(b^d) for BFS. Complete and optimal.Dijkstra's algorithm (Uniform-cost search): actions have different costs. DFS (tree-like search): efficient and complete in finite state spaces. May get stuck in an infinite loop in cyclic state spaces. much smaller needs for space. IDS: fix shortcoming of DFS. Bidirectional search: simultaneously search forward from the initial state and backwards from the goal state, hoping the two searches will meet.   \n\n## Module 3 Lecture Notes\n Problem solving agent - goal; search problem (states, actions, performance measure) and solution. A problem is formulated by States, initial state, actions, transition model, path cost (performance measure), goal test. A solution is optimal if no solution has a lower path cost. State space: the set of all states reachable from the initial state by any sequence of actions. Frontier: those states that are available for expanding (for applying legal actions to). Tree search: treats different paths to the same node as distinct. A node is a data structure constituting part of a search tree. Tree search problem: repeated states. Graph search: check to see if a node has been visited before adding to search queue. must keep track of all possible states (can use a lot of memory). Uninformed search: all non-goal nodes in frontier looks equally good. Strategy: order of tree expansion (LIFO, FIFO, PriorityQ). b: max branching factor, d: depth of the shallowest goal node; m: max length of any path in the state space. BFS: explore shallowest node first. frontier is FIFO. node only inserted into queue after testing to see if it is a goal state. DFS: expand deepest unexpanded node, frontier is LIFO. Use DFS if space limited, many possible solutions with long paths; use BFS if possible infinite paths, some solutions have short paths\n \n\n## Module 3 Definitions\n* **Problem-solving agent** - an agent that can formulate a plan via search\n\n\n# Module 4: Informed Search\n## AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\n heuristic function h(n)=estimated cost of the cheapest path from the state at node n to a goal state. Greedy best-first search: f(n)=h(n). Greedy is completed in finite state spaces but not in infinite ones. A* search: f(n)=g(n)+h(n). g(n) is the path cost from the initial state to node n. f(n) is estimated cost of the best path that continues from n to goal state. Admissible heuristic: never overestimate the cost to reach a goal (optimistic). A heuristic is consistent if, for every node n and every successor n' of n generated by an action a, we have h(n)<=c(n,a,n')+h(n'). triangle inequality. Consistency implies admissibility. With a consistent heuristic A* is optimal. The first time a state is reached it will be on an optimal path, so never need to re-add a state to the frontier and never have to change an entry in reached set. Space complexity of A* is an issue. Iterative-deepening A*: cutoff is the f-cost (g+h). w/o requirement to keep all reached states in memory at a cost of revisiting some states multiple times. Recursive best-first search (RBFS): optimal if h(n) is admissible, space complexity is linear in the depth of the deepest optimal solution. IDA* and RBFS use too little memory. MA* and SMA*: SMA* works like A* expanding the best leaf until memory is full. Then dropping worst leaf node with the highest f-value. Optimal. \nWeighted A* search: weigh h(n) more heavily expand fewer nodes but sacrificing optimality. Beam search: limit size of the frontier keeping only the k nodes with the best scores. \n\n## AIMA Chapter 3.6 Heuristic Functions\n Effective branching factor b*. N+1=1+b*+(b*)^2+...+(b*)^d. If h2(n) and h1(n) are both admissible and h2(n)>=h1(n) for all n then h2 dominates h1. A* using h2 will not expand more nodes than A* using h1. Domination tranlates to efficiency. We can generate heuristic from relaxed problems. \n\n## Module 4 Lecture Notes\n Uniform cost search: misnomer. g(n): the path cost function. \nUCS: extension of BFS. Expand node with lowest path cost, frontier is PriorityQueue ordered by g(n). \nImportant: tests if a node is a goal state when it is selected for expansion, not when it is added to the frontier. \nBetter idea than UCS: node expansion based on an estimate which includes distance to the goal f(n). Sort frontier queue by this new f(n). \nGreedy; A* search. Greedy Best-First search: select node for expansion with minimal evaluation function f(n)=h(n). Exactly UCS with f(n) replacing g(n). \nCompletely ignores g(n): the cost to get to n. Not optimal. Not complete – can stuck in loops. A* search: Best-known form of best-first search. Expand most promising first. \nf(n)=g(n) + h(n). A heuristic h(n) is admissible if it never overestimates the cost to reach the goal (optimistic). Theorem: If h(n) is admissible, A* using Tree Search is optimal.\nThe cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem. If h2(n) and h1(n) are both admissible and h2(n)>=h1(n) then h2 dominates h1.\nUCS expands equally in all directions. A* expands mainly toward the goal, but does hedge its bets to ensure optimality. The space complexity of A* is O(b^d).  \n\n## Module 4 Definitions\n* **Concept** - definition\n\n\n# Module 5: Adversarial Search and Games\n## AIMA Chapter 5.1 Game Theory\n  \n\n## AIMA Chapter 5.2 Optimal Decisions in Games\n  \n\n## AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\n Heuristic alpha-beta tree search. Due to limited computation time we can cut off search early and apply a heuristic evaluation functions to states, effectively treating nonterminal nodes as if they were terminal. Instead of using UTILITY function we use EVAL which estimates a state's utility. EVAL(state, player) estimates the expected utility of state to player. For terminal states, eval(s,p)=utility(s,p). \n\n## AIMA Chapter 5.5 Stochastic Games\n Stochastic games: expectiminimax value. Chance nodes in addition to MAX and MIN nodes.  \n\n## AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\n agent's preference captured by utility function U(s). Expected utility of an action given the evidence EU(a). Maximum expected utility.  \n\n## AIMA Chapter 16.2 The Basis of Utility Theory\n Orderability: for any two lotteries an agent must either prefer one or indifferent. Transitivity: A>B & B>C then A>C. Continuity: A>B>C then exists p such that pA+(1-p)B=C. Substitutability; Monotonicity: if agent prefers A to B the the agent must prefer the lottery with higher prob for A; Decomposability: compound lotteries can be decomped into simpler ones. \n\n## AIMA Chapter 16.3 Utility Functions\n Your summary here \n\n## Module 5 Lecture Notes\n Simplest game environment: multiagent, static, discrete, fully observable, strategic. \nGames as search: MAX & MIN. Initial state, successor function(list of (move,state) pairs specifying legal moves), terminal test, utility function. \nMAX uses search tree to find next move. Game trees: nodes represent board positions and edges are legal moves. All play is computed from MAX’s vantage point. \nEven levels (MAX) & odd levels (MIN). Evaluation function f(n): evaluates how good a ‘board position’ is based on static features of that board alone. \ne.g. tic-tac-toe: f(n)=#of 3 lengths open for MAX-#open for MIN. In non-terminal state, evaluation function gives an estimate of the expected utility of the game from a given position. \nMinimax Rule: recursive process. For a MAX node, the backed-up value is the maximum of the values of its children. For a MIN node, the backed-up value is the minimum of the values of its children. \nAlpha-Beta pruning: alpha: MAX’s current lower bound on MAX’s outcome (at least). beta: MIN’s current upper bound for MIN’s outcome (at most). \nStop evaluating a branch: when eval a MAX node, a value >= beta is backed-up; when eval a MIN node, a value <= alpha is backed-up. \nUpdate alpha & prune by beta in MAX node; update beta & prune by alpha in MIN node. The effectiveness of alpha-beta pruning depends on the ordering in which the successors \nare examined. A limitation of alpha-beta pruning is the assumption that the leaf evaluations are randomly distributed across last layer of the tree. \nExpectimax search: compute the average score under optimal play. Chance nodes are like min nodes but the outcome is uncertain.  \n\n## Module 5 Definitions\n* **Concept** - definition\n\n\n# Module 6: Constraint Satisfaction Problems\n## AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\n Your summary here \n\n## AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\n Your summary here \n\n## AIMA Chapter 6.3 Backtracking Search for CSPs\n Your summary here \n\n## AIMA Chapter 6.4 Local Search for CSPs\n Your summary here \n\n## AIMA Chapter 6.5 The Structure of Problems\n Your summary here \n\n## Module 6 Lecture Notes\n CSP: specialized for identification problems. The goal itself is important, not the path. All paths at the same depth (for some formulations). Contrary to planning problems where the path to the goal is the important thing and paths have various costs, depths. A CSP consists of: Finite set of variables X1, X2, …., Xn, with nonempty domain of possible values for each variable D1, D2, …, Dn. Finite set of constraints C1, C2, …, Cm. A state is defined as an assignment of values to some or all variables. A consistent assignment does not violate the constraints. An assignment is complete when every variable is assigned a value. A solution to a CSP is a complete, consistent assignment. CSP representation: Constraint graph - nodes are variables and arcs are (binary) constraints. Types of CSPs: discrete variables. 1. finite domains O(d^n). 2. infinite domains: integers, strings. job scheduling, variables are start/end days for each job. Continuous variables: start/end times for Hubble Space Telescope observations. linear constraints solvable in polynomial time by linear programming. Types of constraints: unary, binary, higher-order (constraints involve 3 or more variables). preference (soft constraints). e.g. red is better than green. How to solve CSP? Idea 1 as a search problem. Solution is always found at depth n, for n variables (can use DFS).  n variables of domain size d. Branching factor at the root is n*d. Branching factor at next level is (n-1)*d. Tree has n!*d^n leaves. The variable assignments are commutative. e.g, [ step 1: WA = red; step 2: NT = green ] is equivalent to [ step 1: NT = green; step 2: WA = red ]. Therefore, a tree search, not a graph search. Only need to consider assignments to a single variable at each node. b = d and there are d^n leaves (n variables, domain size d ). DFS for CSPs with single-variable assignments is called backtracking search. basic uninformed algorithm for CSPs. Idea 2 improving backtrack efficiency. Heuristics: 1. Which variable should be assigned next? Most constrained variable & (if ties:) Most constraining variable. minimum remaining values (MRV) heuristic. 2. In what order should that variable’s values be tried? Least constraining value. 3. Use Forward checking. Keep track of remaining legal values for unassigned variables. Terminate search when any unassigned variable has no remaining legal values. Constraint propagation goes beyond forward checking & repeatedly enforces constraints locally. Idea 3 (big idea): Inference in CSPs. CSP solvers combine search and inference. Node consistency: satisfies unary constraints. Arc consistency: satisfies binary constraints. (Xi is arc-consistent w.r.t. Xj if for every value v in Di, there is some value w in Dj that satisfies the binary constraint on the arc between Xi and Xj). Edges to arcs: for a connected pair of nodes Xi and Xj , there are two arcs that connect them: (i,j) and (j,i). Given d values, checking arc (i,j) takes O(d^2) time worst case. Waltz/Mackworth towards AC-3. AC-3 complexity: worst case. all nodes can be connected to every other node. so total # of arcs is 2*n*(n-1), i.e. O(n^2). If there are d values, checking arc (i,j) takes O(d^2) time. Each arc (i,j) can only be inserted into the queue d times. Worst case complexity: O(n^2d^3). For planar constraint graphs, the number of arcs can only be linear in N and the time complexity is only O(nd^3). Beyond binary constraints: path consistency; global constraints. in Sudoko, all numbers in a row must be different. Backjumping: Improved backtracking. Fix using conflict-directed backjumping. Go back to predecessors of conflict set.\n\n## Module 6 Definitions\n* **Concept** - definition\n\n\n# Module 7: Logical Agents\n## AIMA Chapter 7.1 Knowledge-Based Agents\n Your summary here \n\n## AIMA Chapter 7.2 The Wumpus World\n Your summary here \n\n## AIMA Chapter 7.3 Logic\n Your summary here \n\n## AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\n Your summary here \n\n## AIMA 7.5 Propositional Theorem Proving\n Your summary here \n\n## Module 7 Lecture Notes\n Logical agents: Knowledge-based agents use a process of reasoning over an internal representation of knowledge to decide what action to take. A KB contains a set of sentence that are written in a knowledge representation language. The sentence contains some assertion about the world. Axioms and derived sentences. The process of deriving new sentences from old sentences is called inference. In addition to asserting new knowledge into its KB, a knowledge-based agent can also query the KB and ask it to derive new knowledge in order to select what action it should take. Wampa world: Deterministic, discrete, static, single-agent, sequential and partially-observable. Models are mathematical abstractions that have a fixed set of truth values which are {true, false} for each sentence. If sentence a is true in model m then we say m satisfies a or m is a model of a. M(a) means the set of all models of a. Logical reasoning involves the entailment relation between sentence. entailment is the idea that a sentence follows logically from another sentence. alpha entails beta iff M(alpha) is subset of M(beta). alpha is more specific, or stronger than beta. Possible Worlds is the process of enumerating all Possible Worlds that are compatible with the KB. The consequences of a KB is a large set of additional sentences that are entailed given the sentences that have been added to the KB. Design inference algorithms to enumerate these sentences. Sentence alpha is derived from KB with algorithm i. Propositional logic: atomic sentences are represented with a single propositional symbol. Propositional symbols stand for a statement that can be true or false. e.g. W13 stands for there is a Wampa at location [1,3]. Soundness – the inference algorithm should only derive entailed sentences. Completeness – an inference algorithm is complete if it can derive all sentences that are entailed. Monotonicity: we can safely add to the KB, without invalidating anything else that we inferred. Inference rules: And Elimination; Bidirectional Elimination; Modus Ponens. There is a single inference rule, resolution, which yields a complete inference algorithm when coupled with any complete search algorithm. The resolution rule only applies to clauses (disjunction of literals). Every sentence of propositional logic is equivalent to a conjunction of clauses. CNF. Inference procedures based on resolution work using the principle of proof by contradiction. To show KB entails alpha we show {KB and not(alpha)} is not unsatisfiable. First, convert {KB and not(alpha)} into CNF. Then apply the resolution rule to the resulting clauses. Resolve every pair that contains complementary literals to produce a new clause. If that new clause isn’t already present, then add it to the set. Continue the process until one of two things happens: 1. There are no new clauses that can be added. In this case, the KB does not entail alpha. 2. Two clauses resolve to yield the empty clause, in which case the KB does entail alpha. The resolution algorithm is complete. Definite clause – disjunction of literals where exactly one is positive. Horn clause – disjunction of literals where at most one is positive. Horn clauses are closed under resolution. If you apply resolution to a horn clause, you get back a horn clause. Inference with Horn clauses can be done through the forward-chaining and backward-chaining algorithms. Deciding entailment with Horn clauses can be done in linear time with respect to the size of the KB. Forward Chaining is a kind of data-driven reasoning. An agent can start with known data, adding premises as new percepts come in, and then incrementally apply the forward chaining algorithm to derive new conclusions. It doesn’t have to have a specific query in mind to do so. Backward chaining is a form of goal-directed reasoning by working backwards from a query. If the query is known to be true, then no work is needed. Otherwise, the algorithm finds implications in the KB whose conclusions is the query. If all the premises of one of those can be proved to be true (via backward chaining), then the query is true. \n\n## Module 7 Definitions\n* **Concept** - definition\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    }
  ]
}