{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of Copy of FINAL Template_for CIS521_midterm notes.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "brNDGX-kvzds"
      },
      "source": [
        "# Template for Notes for CIS 521 Midterm 1\n",
        "\n",
        "The midterm exam for CIS 521 will be **open notes**. No other matierals are allowed for the exam - you may not use the textbook, the internet, a Python envionement, or communicate with anyone during the exam. \n",
        "\n",
        "This python notebook is where you should save all of your notes.  It provides a structured template that allows you to add summaries for each of the textbook sections that are part of the required reading, plus room for lecture notes and definitions.\n",
        "\n",
        "Your summaries must be written in your own words.  Do not copy and paste from the textbook.  Do not share your notes with classmates, or copy any of your classmates' notes.  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aPtM8g7R0Pds"
      },
      "source": [
        "# Instructions\n",
        "\n",
        "1. Make a copy of this Google Colab file. From the colab menu, pick **File > Save a copy in Drive**.  That will save a copy of this notebook in your Google drive.\n",
        "2. Go through each section of the notebook, and fill in all the summaries for each chapter and its corresponding sections.  Be sure to save your work frequently!\n",
        "3. Read the notes below about how you can format your summaries.\n",
        "4. In addition to these notes, you are allowed 1 page (both sides) for figures or handwritten notes.\n",
        "5. Submit your version of this notebook to Gradescope before you start the exam.\n",
        "6. If you are taking the exam in person, then you must print a copy of your notes for yourself.   Here's how to print: \n",
        "* Run the last cell of this notebook\n",
        "* Click the \"Mirror cell in tab\" button (this is located in the upper right corner of the cell, immediately to the right of the trash can).  This will open the cell.  \n",
        "* In Chrome, you can then print your notes using the **File > Print** option from the browser's menu.  \n",
        "* In the print options, change the layout from Landscape to Portrait. \n",
        "* Clicking the Save, will save a PDF that you can then print.\n",
        "\n",
        "## Formatting Notes\n",
        "\n",
        "1. The max length for each section's summary is 500 words / 3000 characters.\n",
        "2. You can also write a 2000 word summary of the lectures for each module.\n",
        "3. You may also write definitions of concepts for each module.  The definitions should be written in your own words (not copy and pasted), and they should be no more than 20 words long for each concept. \n",
        "2. You can use Markdown formatting for your summaries. \n",
        "3. You can use Latex formatting for equations. Put two dollar signs before and after your equation like this: \n",
        "$$ \\epsilon = \\frac{1}{mc^2} $$\n",
        "4. When you write python code or psuedocode algorithms, you should surround it with triple backticks like this:\n",
        "```\n",
        "def function():\n",
        "      print('hello world')\n",
        "```\n",
        "5. Write \\ whenever you want to keep writing on the next line without a newline being inserted."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cw5IpDLCZa4W"
      },
      "source": [
        "## Example summary\n",
        "\n",
        "This is a sample summary of [Section 2.1.1](https://web.stanford.edu/~jurafsky/slp3/2.pdf ) of Jurfasky and Martin's Speech and Language Processing's book\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lzaavW96tcxM",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e57efcbd-6799-4c96-d076-2286bbc81f1a"
      },
      "source": [
        "chapter_2_1_example = \"\"\"\n",
        "Regular expressions are an algebraic notation for characterizing \\\n",
        "a set of strings that are useful for searching for a pattern or through a corpus.\\\n",
        "Only the first matching result will be shown. The search string is case sensitive, \\\n",
        "but it can consist of a single character or a sequence of characters. \\\n",
        "While square braces specify a disjunction of characters, the dash can more easily look \\\n",
        "for characters over a range. The caret specifies the non-existence of a character. \\\n",
        "Additionally, the (?) is used to for flexibility on a single character, meaning \\\n",
        "\"the preceding character or nothing\". More importantly, the Kleene star means zero or \\\n",
        "more occurrences of the immediately previous character or regular expression in square braces.\\\n",
        "...\n",
        "\"\"\"\n",
        "\n",
        "length = len(chapter_2_1_example.split(\" \"))\n",
        "print(\"The example summary is {len} words long, so you could write an additional \\\n",
        "{remaining} words.\".format(len=length, remaining=(500-length)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The example summary is 116 words long, so you could write an additional 384 words.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s3Uk9gQTevcj"
      },
      "source": [
        "## Example summary length\n",
        "\n",
        "The maximum summary length for each section is 500 words / 3000 characters.  Hopefully you'll find that to be quite generous. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8DvZ57A-djo1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "de162191-ba65-42d0-e3c8-15518a4d0453"
      },
      "source": [
        "# This is an example of a 500 word placeholder\n",
        "placeholder = \"\"\"ultrices vitae auctor eu augue ut lectus arcu bibendum at varius \\\n",
        " vel pharetra vel turpis nunc eget lorem dolor sed viverra ipsum nunc aliquet \\\n",
        " bibendum enim facilisis gravida neque convallis a cras semper auctor neque vitae \\\n",
        " tempus quam pellentesque nec nam aliquam sem et tortor consequat id porta nibh \\\n",
        " venenatis cras sed felis eget velit aliquet sagittis id consectetur purus ut \\\n",
        " faucibus pulvinar elementum integer enim neque volutpat ac tincidunt vitae semper \\ \n",
        " quis lectus nulla at volutpat diam ut venenatis tellus in metus vulputate eu \\\n",
        " scelerisque felis imperdiet proin fermentum leo vel orci porta non pulvinar neque \\ \n",
        " laoreet suspendisse interdum consectetur libero id faucibus nisl tincidunt eget \\\n",
        " nullam non nisi est sit amet facilisis magna etiam tempor orci eu lobortis \\\n",
        " elementum nibh tellus molestie nunc non blandit massa enim nec dui nunc mattis \\\n",
        " enim ut tellus elementum sagittis vitae et leo duis ut diam quam nulla porttitor \\\n",
        " massa id neque aliquam vestibulum morbi blandit cursus risus at ultrices mi tempus \\\n",
        " imperdiet nulla malesuada pellentesque elit eget gravida cum sociis natoque \\\n",
        " penatibus et magnis dis parturient montes nascetur ridiculus mus mauris vitae \\\n",
        " ultricies leo integer malesuada nunc vel risus commodo viverra maecenas accumsan \\\n",
        " lacus vel facilisis volutpat est velit egestas dui id ornare arcu odio ut sem nulla \\\n",
        " pharetra diam sit amet nisl suscipit adipiscing bibendum est ultricies integer \\\n",
        " quis auctor elit sed vulputate mi sit amet mauris commodo quis imperdiet massa \\\n",
        " tincidunt nunc pulvinar sapien et ligula ullamcorper malesuada proin libero nunc \\\n",
        " consequat interdum varius sit amet mattis vulputate enim nulla aliquet porttitor \\\n",
        " lacus luctus accumsan tortor posuere ac ut consequat semper viverra nam libero \\\n",
        " justo laoreet sit amet cursus sit amet dictum sit amet justo donec enim diam \\\n",
        " vulputate ut pharetra sit amet aliquam id diam maecenas ultricies mi eget mauris \\\n",
        " pharetra et ultrices neque ornare aenean euismod elementum nisi quis eleifend quam \\\n",
        " adipiscing vitae proin sagittis nisl rhoncus mattis rhoncus urna neque viverra justo \\\n",
        " nec ultrices dui sapien eget mi proin sed libero enim sed faucibus turpis in \\\n",
        " eu mi bibendum neque egestas congue quisque egestas diam in arcu cursus euismod \\\n",
        " quis viverra nibh cras pulvinar mattis nunc sed blandit libero volutpat sed cras \\\n",
        " ornare arcu dui vivamus arcu felis bibendum ut tristique et egestas quis ipsum \\\n",
        " suspendisse ultrices gravida dictum fusce ut placerat orci nulla pellentesque \\\n",
        " dignissim enim sit amet venenatis urna cursus eget nunc scelerisque viverra mauris \\\n",
        " in aliquam sem fringilla ut morbi tincidunt augue interdum velit euismod in \\\n",
        " pellentesque massa placerat duis ultricies lacus sed turpis tincidunt id aliquet \\\n",
        " risus feugiat in ante metus dictum at tempor commodo ullamcorper a lacus vestibulum \\\n",
        " sed arcu non odio euismod lacinia at quis risus sed vulputate odio ut enim blandit \\\n",
        " volutpat maecenas volutpat blandit aliquam etiam erat velit scelerisque in \\\n",
        " dictum non consectetur a erat nam at lectus urna\"\"\"\n",
        "\n",
        "print(\"The length in words is\", len(placeholder.split(' ')))\n",
        "print(\"The length in characters is\", len(placeholder))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 500\n",
            "The length in characters is 3030\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5yG3-YncJfTB"
      },
      "source": [
        "# Permissions to use your summaries\n",
        "\n",
        "We'd like to use your summaries to help develop better educational resources like [remember.school](https://remember.school) or a question-answering system about the textbook.  \n",
        "\n",
        "1. Please enter your name.  \n",
        "\n",
        "2. If you grant permission for us to use your summaries under a [Creative Commons License](https://creativecommons.org/licenses/by/4.0/), please set ```permission = True```.  If you do not grant permission, please set it to false. Your answer won't affect your grade.\n",
        "\n",
        "3. If you grant us permission, and you'd like to remain anonymous, set ```anonymous = True```.  We'll remove your name before sharing your summaries with anyone. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2RECnJArJew4"
      },
      "source": [
        "name = \"\"\n",
        "permission = True\n",
        "anonymous = True"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GZZXyrV-xuAp"
      },
      "source": [
        "# Module 1 - Intro to AI"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y2NfE1CKxzS-"
      },
      "source": [
        "# Russell and Norvig, AIMA 4th Edition, Chapter 1 “Introduction”, Section 1.1 “What is AI”\n",
        "\n",
        "chapter1_1 = \"\"\"A brief overview of the field of A.I. and the key terminology underlying the study of A.I. including rationality, thinking, logic, and intelligence. Proposed the Turing test. Composed of NLP (communicate like human), Knowledge storing and representation (from sensors), automated reasoning (drawing conclusions), and Machine Learning (adapting\n",
        "to new circumstance and extracting patterns) and extended Turing Test (How entity perceives, aka computer vision, and manipulation of environment, aka robotics/actuators). \n",
        "The idea of what it means to think humanly and logic itself were presented, the latter begining with Syllogisms and needing a complete/total knowledge base (which is impractical).\n",
        "Now logic encompasses probability theory, allowing the making of decisions even when the world is not completely observable or uncertainty exists (nondeterministic, stochastic task environments)\"\"\" \n",
        "\n",
        "# Russell and Norvig, AIMA 4th Edition, Chapter 27 “Philosophical Foundations”, Sections 27.1 “The Limits of AI” and 27.2 “Can Machines Really Think?”\n",
        "chapter27_1 = \"\"\" Limits of AI and initial arguments against AI including an argument from informality, that there are too many informal rules in reality\n",
        "and too difficult to quanitifed/represented in a knowledge base. This is sort of true, giving rise to the qualification problem, probability theory is supposedly a solution. Essentially, AI does not offer complete answers and therefore is impossible.\n",
        "Another argument is from disability. The only understanding is that programs can't be exactly human (because they are not). Mathematical objection is another claim against AI. Argument is that \n",
        "AI is limited by Godel's incompleteness theorem, being that machinese are formal systems while humans are not. However, incompleteness only applies to math, not computers which are finite (just like people). Computers != turing machines\"\"\"\n",
        "chapter27_2 = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Russell and Norvig, AIMA 4th Edition, Chapter 2 “Intelligent Agents” (whole chapter)\n",
        "chapter2_1 = \"\"\" Deep dive into what makes an agent intelligent. Agent is intelligent if it perceives env. through sensors and acts through its actuators. The actions it takes is limited to percept sequence as agent function maps percept sequence to actions.\n",
        "Thus, an agent cannot do something that it does not know is possble (not in its set of actions). Also discusses the agent function and agent program, the prior being a external mathematical abstraction while the latter is a internal implementation of the former.\n",
        " \"\"\"\n",
        "chapter2_2 = \"\"\" Focus on the idea of rationality and what is good behavior. Specifically, that pure consequentialism (evaluating behaviour purely on outcome) is insufficient. Introduced the notion of a performance measure, which we later know to be\n",
        "a key component of the PEAS-agent building framework. Rational decision-making requires 4 components: the perfomrnace measure, prior knowledge of the task env, set of actions that can be performed, and percept sequence up until the decision point.\n",
        "This gives rise to the definition of a rational agent, an agent which selects and performs the action that is expected to achieve best possible outcome of a prespecified performance measure for each\n",
        "possible percept sequence that is built off of previous percept sequence (to date) and built in knowledge/learning.\n",
        "\"\"\"\n",
        "chapter2_3 = \"\"\" Discussion of the task environment, which is where the problem is. Well-designing agents often rely on description of task environment amap. Environments can be partially or fully observable, and observability\n",
        "is discussed in relation to how much of the environment the agent's sensors can perceive. An unobservable environment is one in which an agent has no sensors (or relevant sensors).\n",
        "Types of environments: Single/multi-agent, deterministic/non-deterministic/stochastic, episodic/sequential, static/dynamic, discrete/continuous, (known/unknown).\n",
        "\n",
        "Deterministic environments are one where the environment is only impacted by the actions of the agent. Non-deterministic and stochastic both relate to uncertainty, though stochastic adds probability to outcome. \n",
        "Note that knowing != observability. I can know the environment but have partial observability (think Solitaire).\n",
        "Most difficult environments are those like reality, multi-agent, non-deterministic/stochastic, sequential, dynamic, continuous, and unknown.\n",
        " \"\"\"\n",
        "chapter2_4 = \"\"\" Desigining and structuring agents - think PEAS. The types of agents are simple reflex, model-based, goal-based, and utility-based. There are also extension types being model-free and learning agent.\n",
        "The learning agent involves a loop involving the critic (evaluate performance in relation to PM), learning element, problem, generator, and performance element. Also discusses the different representations of the world being\n",
        "Atomic (think basic search algorithms, informed/uninformed ones), Factored (think CSP algs inc. AC3), and Structured. Physical memory can also be represented in a localist and distributed manner.\"\"\"\n",
        "\n",
        "# Lecture notes \n",
        "module1_lecture_notes = \"\"\" PEAS for agent design: Performance Measure, (task) environment, actuators, sensors \"\"\"\n",
        "\n",
        "\n",
        "# Definitions of concepts from Module 1.\n",
        "# You can define any bolded term in the textbook, \n",
        "# but you most use your own words for the definitions.\n",
        "# Each definition should be at most 20 words long.\n",
        "module1_definitions = {\n",
        "\"Agent\" : \"Anything that perceives its environment through sensors and interacts with environment through actuators\",\n",
        "\"Percept\" : \"content received by sensors\",\n",
        "\"Percept Sequence\": \"Entire history of percept that an agent has received\",\n",
        "\"Agent Function\": \"Mathematical abstraction to map percept sequence to set of actions, external\",\n",
        "\"Agent Program\": \"Internal implementation of agent function\",\n",
        "\"Intelligence\" : \"How one thinks and acts\",\n",
        "\"A.I.\": \"The understanding and building of intelligent entities\",\n",
        "\"Rationality\": \"Doing the right thing, given the objective provided/specified. Rationality is reliant on action, not only thought.\",\n",
        "\"Introspection\": \" Ability to catch one's own thoughts\",\n",
        "\"Psychological Experiments\": \"Having thoughts that are observable through experimentation\",\n",
        "\"Brain Imaging\": \" Physically being able to watch a brain (not necessarily biological) in action\",\n",
        "\"Rational Agent\": \"Agent that acts to achieve best possible (rational) or best expected outcome (limited rationality)\",\n",
        "\"Standard Model\": \"Doing the right thing based on the predefined/specified objective, ex: Max or Min value\",\n",
        "\"Value Alignment Problem\": \"Simply achieving objective may not be desirable, the problem of aligning objective to true preferences\",\n",
        "\"Omniscience\": \"Knowing the eventual outcome and acting accordingly, there is no uncertainty. Essentially allows for perfection which != rationality.\",\n",
        "\"Information Gathering\": \"Doing actions in order to augment future percepts\",\n",
        "\"Autonomy\": \"Agent being able to learn based on own percepts and learning processes independent from creator/designed.\",\n",
        "\"Limited Rationality\": \"When agent achieves the best expected value\",\n",
        "\"Weak AI\": \"Machines acting as if intelligent, but not thinking curiously\",\n",
        "\"Strong AI\" :\"Consciously thinking machines\",\n",
        "\n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "elCCgY64spuq"
      },
      "source": [
        "## Preview of your notes for Module 1\n",
        "\n",
        "Here's a preview of your how your notes will be displayed.  At the end of this Python notebook, we'll aggregate your notes for all modules together into a single display.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gYYeNcdM_maE",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "cellView": "form",
        "outputId": "38b10c7a-cda1-4535-fc43-322c2fd7bfb0"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "from IPython.display import display, Markdown, Latex\n",
        "\n",
        "module1_sections = [\n",
        "  (\"AIMA Chapter 1.1 What is AI?\", chapter1_1),\n",
        "  (\"AIMA Chapter 27.1 The Limits of AI\", chapter27_1),\n",
        "  (\"AIMA Chapter 27.1 Can Machines Really Think?\", chapter27_2),\n",
        "  (\"AIMA 2.1 Agents and Environments\", chapter2_1),\n",
        "  (\"AIMA 2.2 Good Behavior: The Concept of Rationality\", chapter2_2),\n",
        "  (\"AIMA 2.3 The Nature of Environments\", chapter2_3),\n",
        "  (\"AIMA 2.4 The Structure of Agents\", chapter2_4),\n",
        "  (\"Module 1 Lecture Notes\", module1_lecture_notes),\n",
        "]\n",
        "\n",
        "\n",
        "def length_check(sections):\n",
        "  warnings = \"\"\n",
        "  for section, summary in sections:\n",
        "    max_word_length = 500\n",
        "    max_character_length = 3000\n",
        "    if \"lecture\" in section.lower():\n",
        "      # It's OK to do longer summaries of the lectures\n",
        "      max_word_length = 2000\n",
        "      max_character_length = 12000  \n",
        "    if (len(summary) > max_character_length or len(summary.split(' ')) > max_word_length):\n",
        "      warnings += \"WARNING: Your summary of '{section}' is too long!\\n\".format(section=section)\n",
        "  return warnings\n",
        "\n",
        "def length_check_definitions(definitions):\n",
        "  warnings = \"\"\n",
        "  for concept, definition in definitions.items():\n",
        "    if(len(definition.split(' ')) > 20):\n",
        "      warnings += \"WARNING: Your definition of '{concept}' is too long!\\n\".format(concept=concept)\n",
        "  return warnings\n",
        "\n",
        "\n",
        "def format_module(module_num, title, sections, definitions, show_placeholder=False):\n",
        "  markdown = \"\"\n",
        "\n",
        "  # Check summary lengths\n",
        "  warnings = length_check(sections)\n",
        "  if warnings != \"\":\n",
        "    markdown += \"## \" + warnings\n",
        "\n",
        "  # Check definition lengths\n",
        "  warnings = length_check_definitions(definitions)\n",
        "  if warnings != \"\":\n",
        "    markdown += \"## \" + warnings\n",
        "\n",
        "  markdown += \"# Module {num}: {title}\\n\".format(num=module_num, title=title)\n",
        "\n",
        "\n",
        "  for section, summary in sections:\n",
        "    if show_placeholder and summary == \" Your summary here \":\n",
        "      summary = placeholder\n",
        "    markdown += \"## {section}\\n{summary}\\n\\n\".format(section=section, summary=summary)\n",
        "  \n",
        "  markdown += \"## Module {num} Definitions\\n\".format(num=module_num)\n",
        "  for concept, definition in definitions.items():\n",
        "    markdown += \"* **{concept}** - {definition}\\n\".format(concept=concept.capitalize(), definition=definition)\n",
        "  return markdown\n",
        "\n",
        "\n",
        "module1_markdown = format_module(1, \"Rational Agents and Task Environments\", module1_sections, module1_definitions)\n",
        "display(Markdown(module1_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 1: Rational Agents and Task Environments\n## AIMA Chapter 1.1 What is AI?\nA brief overview of the field of A.I. and the key terminology underlying the study of A.I. including rationality, thinking, logic, and intelligence. Proposed the Turing test. Composed of NLP (communicate like human), Knowledge storing and representation (from sensors), automated reasoning (drawing conclusions), and Machine Learning (adapting\nto new circumstance and extracting patterns) and extended Turing Test (How entity perceives, aka computer vision, and manipulation of environment, aka robotics/actuators). \nThe idea of what it means to think humanly and logic itself were presented, the latter begining with Syllogisms and needing a complete/total knowledge base (which is impractical).\nNow logic encompasses probability theory, allowing the making of decisions even when the world is not completely observable or uncertainty exists (nondeterministic, stochastic task environments)\n\n## AIMA Chapter 27.1 The Limits of AI\n Limits of AI and initial arguments against AI including an argument from informality, that there are too many informal rules in reality\nand too difficult to quanitifed/represented in a knowledge base. This is sort of true, giving rise to the qualification problem, probability theory is supposedly a solution. Essentially, AI does not offer complete answers and therefore is impossible.\nAnother argument is from disability. The only understanding is that programs can't be exactly human (because they are not). Mathematical objection is another claim against AI. Argument is that \nAI is limited by Godel's incompleteness theorem, being that machinese are formal systems while humans are not. However, incompleteness only applies to math, not computers which are finite (just like people). Computers != turing machines\n\n## AIMA Chapter 27.1 Can Machines Really Think?\n Your summary here \n\n## AIMA 2.1 Agents and Environments\n Deep dive into what makes an agent intelligent. Agent is intelligent if it perceives env. through sensors and acts through its actuators. The actions it takes is limited to percept sequence as agent function maps percept sequence to actions.\nThus, an agent cannot do something that it does not know is possble (not in its set of actions). Also discusses the agent function and agent program, the prior being a external mathematical abstraction while the latter is a internal implementation of the former.\n \n\n## AIMA 2.2 Good Behavior: The Concept of Rationality\n Focus on the idea of rationality and what is good behavior. Specifically, that pure consequentialism (evaluating behaviour purely on outcome) is insufficient. Introduced the notion of a performance measure, which we later know to be\na key component of the PEAS-agent building framework. Rational decision-making requires 4 components: the perfomrnace measure, prior knowledge of the task env, set of actions that can be performed, and percept sequence up until the decision point.\nThis gives rise to the definition of a rational agent, an agent which selects and performs the action that is expected to achieve best possible outcome of a prespecified performance measure for each\npossible percept sequence that is built off of previous percept sequence (to date) and built in knowledge/learning.\n\n\n## AIMA 2.3 The Nature of Environments\n Discussion of the task environment, which is where the problem is. Well-designing agents often rely on description of task environment amap. Environments can be partially or fully observable, and observability\nis discussed in relation to how much of the environment the agent's sensors can perceive. An unobservable environment is one in which an agent has no sensors (or relevant sensors).\nTypes of environments: Single/multi-agent, deterministic/non-deterministic/stochastic, episodic/sequential, static/dynamic, discrete/continuous, (known/unknown).\n\nDeterministic environments are one where the environment is only impacted by the actions of the agent. Non-deterministic and stochastic both relate to uncertainty, though stochastic adds probability to outcome. \nNote that knowing != observability. I can know the environment but have partial observability (think Solitaire).\nMost difficult environments are those like reality, multi-agent, non-deterministic/stochastic, sequential, dynamic, continuous, and unknown.\n \n\n## AIMA 2.4 The Structure of Agents\n Desigining and structuring agents - think PEAS. The types of agents are simple reflex, model-based, goal-based, and utility-based. There are also extension types being model-free and learning agent.\nThe learning agent involves a loop involving the critic (evaluate performance in relation to PM), learning element, problem, generator, and performance element. Also discusses the different representations of the world being\nAtomic (think basic search algorithms, informed/uninformed ones), Factored (think CSP algs inc. AC3), and Structured. Physical memory can also be represented in a localist and distributed manner.\n\n## Module 1 Lecture Notes\n PEAS for agent design: Performance Measure, (task) environment, actuators, sensors \n\n## Module 1 Definitions\n* **Agent** - Anything that perceives its environment through sensors and interacts with environment through actuators\n* **Percept** - content received by sensors\n* **Percept sequence** - Entire history of percept that an agent has received\n* **Agent function** - Mathematical abstraction to map percept sequence to set of actions, external\n* **Agent program** - Internal implementation of agent function\n* **Intelligence** - How one thinks and acts\n* **A.i.** - The understanding and building of intelligent entities\n* **Rationality** - Doing the right thing, given the objective provided/specified. Rationality is reliant on action, not only thought.\n* **Introspection** -  Ability to catch one's own thoughts\n* **Psychological experiments** - Having thoughts that are observable through experimentation\n* **Brain imaging** -  Physically being able to watch a brain (not necessarily biological) in action\n* **Rational agent** - Agent that acts to achieve best possible (rational) or best expected outcome (limited rationality)\n* **Standard model** - Doing the right thing based on the predefined/specified objective, ex: Max or Min value\n* **Value alignment problem** - Simply achieving objective may not be desirable, the problem of aligning objective to true preferences\n* **Omniscience** - Knowing the eventual outcome and acting accordingly, there is no uncertainty. Essentially allows for perfection which != rationality.\n* **Information gathering** - Doing actions in order to augment future percepts\n* **Autonomy** - Agent being able to learn based on own percepts and learning processes independent from creator/designed.\n* **Limited rationality** - When agent achieves the best expected value\n* **Weak ai** - Machines acting as if intelligent, but not thinking curiously\n* **Strong ai** - Consciously thinking machines\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9xRbdqDox2zD"
      },
      "source": [
        "#Module 2 - Python"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DpBvu7EMx7sy"
      },
      "source": [
        "# No readings for this module.  Instead you can summarize the lecture instead.\n",
        "\n",
        "# Lecture notes \n",
        "module2_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Each definition should be at most 20 words long.\n",
        "module2_definitions = {\n",
        "\"mutability\" : \"your definition here\",\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ydOK__lwWgdp",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 209
        },
        "outputId": "25d3f35b-28e9-40f2-ae2e-eace6848ae08"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "\n",
        "module2_sections = [\n",
        "  (\"Module 2 Lecture Notes\", module2_lecture_notes),\n",
        "]\n",
        "\n",
        "module2_markdown = format_module(2, \"Python\", module2_sections, module2_definitions)\n",
        "display(Markdown(module2_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 2: Python\n## Module 2 Lecture Notes\n Your summary here \n\n## Module 2 Definitions\n* **Mutability** - your definition here\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "46MQLR-Gx8dZ"
      },
      "source": [
        "#Module 3 - Search Problems"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BN0b1eVYx-nd"
      },
      "source": [
        "# Russell and Norvig, AIMA 4th Edition, AIMA Chapter 3 “Problem Solving by Search” (3.1-3.4)\n",
        "chapter3_1 = \"\"\" Problem Solving Agents. Process of goal solving is goal formulation (destination state), problem formation, search, and execution. World is represented atomically, and search algorithms are either informed or uninformed.\n",
        "Search problems consist of 6 components: Initial state, a state space, goal state, actions, transition model, and action cost function. \n",
        " \"\"\"\n",
        "chapter3_2 = \"\"\" Discusses standardized problems, which are hypothetical problems/environments, and real-world problems. Example of standardized problem is grid world, real-world problem is route-finding. \"\"\"\n",
        "chapter3_3 = \"\"\" Introduces algorithms include: Best first search (choose node with minimum eval function value f(n). Best first search uses a priority queue. Graph search is if a search algorithm checks for redundant paths, otherwise algorithm is a\n",
        "tree-like search. Completeness is dependent on the systematic way an algorithm works. Problems may arise in infinite search spaces. d = depth, m = max actions in any path, b is branching factor (number of successors per node\"\"\"\n",
        "chapter3_4 = \"\"\" Overview of uninformed search algorithms. BFS (queue), Dijkstra's/UCS (exact opposite of what name suggests), DFS (stack, memory constraint, not complete or optimal), DLS (same problem as DFS, but limit depth),\n",
        "IDDFS (mix of BFS and DFS, O(b^d) or, O(b^m) if no solution, does not store previous states in memory, just regenerates them), Bidirectional search (search from initial to goal and goal to initial concurrently, 2 frontiers, 2 reached states, solution when frontier meet.\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module3_lecture_notes = \"\"\" We deal the most basic of environments for basic search algorithms: Epsidoci, fully-observable, discrete, static, deterministic, and known. If environment is unknown, execute action at random.\"\"\"\n",
        "\n",
        "# Definitions of concepts from Module 3.\n",
        "# You can define any bolded term in the textbook, \n",
        "# but you most use your own words for the definitions.\n",
        "# Each definition should be at most 20 words long.\n",
        "module3_definitions = {\n",
        "\"problem-solving agent\" : \"an agent that can formulate a plan via search\",\n",
        "\"Uninformed\": \" No estimate of how far agent is from goal state.\",\n",
        "\"Solution\": \"Path from the initial state to goal state that is optimal if is lowest-path cost\",\n",
        "\"Completeness\": \"Answer will be found if it exists, and failure if not\",\n",
        "\"Optimality\": \"Solution will be of lowest path cost\",\n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lvtN3p7ET3Tx",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 678
        },
        "outputId": "2eb864de-bb41-49b0-f37a-f47fbbe868fc"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module3_sections = [\n",
        "  (\"AIMA Chapter 3.1 Problem-Solving Agents\", chapter3_1),\n",
        "  (\"AIMA Chapter 3.2 Example Problems\", chapter3_2),\n",
        "  (\"AIMA Chapter 3.3 Search Algorithms\", chapter3_3),\n",
        "  (\"AIMA Chapter 3.4 Uninformed Search Strategies\", chapter3_4),\n",
        "  (\"Module 3 Lecture Notes\", module3_lecture_notes),\n",
        "]\n",
        "\n",
        "module3_markdown = format_module(3, \"Search Problems\", module3_sections, module3_definitions)\n",
        "display(Markdown(module3_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 3: Search Problems\n## AIMA Chapter 3.1 Problem-Solving Agents\n Problem Solving Agents. Process of goal solving is goal formulation (destination state), problem formation, search, and execution. World is represented atomically, and search algorithms are either informed or uninformed.\nSearch problems consist of 6 components: Initial state, a state space, goal state, actions, transition model, and action cost function. \n \n\n## AIMA Chapter 3.2 Example Problems\n Discusses standardized problems, which are hypothetical problems/environments, and real-world problems. Example of standardized problem is grid world, real-world problem is route-finding. \n\n## AIMA Chapter 3.3 Search Algorithms\n Introduces algorithms include: Best first search (choose node with minimum eval function value f(n). Best first search uses a priority queue. Graph search is if a search algorithm checks for redundant paths, otherwise algorithm is a\ntree-like search. Completeness is dependent on the systematic way an algorithm works. Problems may arise in infinite search spaces. d = depth, m = max actions in any path, b is branching factor (number of successors per node\n\n## AIMA Chapter 3.4 Uninformed Search Strategies\n Overview of uninformed search algorithms. BFS (queue), Dijkstra's/UCS (exact opposite of what name suggests), DFS (stack, memory constraint, not complete or optimal), DLS (same problem as DFS, but limit depth),\nIDDFS (mix of BFS and DFS, O(b^d) or, O(b^m) if no solution, does not store previous states in memory, just regenerates them), Bidirectional search (search from initial to goal and goal to initial concurrently, 2 frontiers, 2 reached states, solution when frontier meet.\n\n\n## Module 3 Lecture Notes\n We deal the most basic of environments for basic search algorithms: Epsidoci, fully-observable, discrete, static, deterministic, and known. If environment is unknown, execute action at random.\n\n## Module 3 Definitions\n* **Problem-solving agent** - an agent that can formulate a plan via search\n* **Uninformed** -  No estimate of how far agent is from goal state.\n* **Solution** - Path from the initial state to goal state that is optimal if is lowest-path cost\n* **Completeness** - Answer will be found if it exists, and failure if not\n* **Optimality** - Solution will be of lowest path cost\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WHREsHSSyA-h"
      },
      "source": [
        "#Module 4 - Informed Search"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tuFC8w7CyCns"
      },
      "source": [
        "# Russell and Norvig, AIMA Chapter 3 “Problem Solving by Search” (3.5-3.6)\n",
        "chapter3_5 = \"\"\" Informed Heuristic Search strategies: Greedy best-first search, speedy search, A*, weighted A*, (search limited by too little memory use) Beam search (limit frontier), IDA*, recursive best-first search (RBFS, uses backed-up value),\n",
        "and capping memory allotment to search algorithms: SMA*, MA*. Also introduced Bidirectional heuristic search, which has front-to-end and front-to-front variations.\n",
        "Introduces idea of admissibility and consistency (related to triangle equality) of heuristic function. A* is key algorithm,\n",
        "particularly great if it's given a consistent heuristic (optimally efficient). A* prunes nodes that do not lead to optimal solution. Improve A* run-time via satisficing solution (allow\n",
        "inadmissible heuristic). \"\"\"\n",
        "chapter3_6 = \"\"\"Domination and heuristic function deep dive. Good way of generating a heuristic includes: using a relaxed problem, using a sub-problem (pattern databases and disjoint pattern databases),\n",
        "using landmark points, shortcuts. \"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module4_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module4_definitions = {\n",
        "\"admissibility\" : \"Admissible is one that does not overstimate actual cost to reach goal\",\n",
        "\"consistency\": \"Cost of current h(n) is less or equal to sum of cost of action + h(n')\",\n",
        "\"monotonic\": \"Path cost does not decrease, but may stay same. Strictly monotonic is only increase.\",\n",
        "\"reference count\": \"Used in MBS, keeps number of times state has been reached. Used to remove states.\",\n",
        "\n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vqtEs0gHs9sO",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 477
        },
        "cellView": "form",
        "outputId": "b2e10731-61d4-4fdd-9d7d-e6add244f7a5"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module4_sections = [\n",
        "  (\"AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\", chapter3_5),\n",
        "  (\"AIMA Chapter 3.6 Heuristic Functions\", chapter3_6),\n",
        "  (\"Module 4 Lecture Notes\", module4_lecture_notes),\n",
        "]\n",
        "\n",
        "module4_markdown = format_module(4, \"Informed Search\", \n",
        "                         module4_sections, \n",
        "                         module4_definitions)\n",
        "display(Markdown(module4_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 4: Informed Search\n## AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\n Informed Heuristic Search strategies: Greedy best-first search, speedy search, A*, weighted A*, (search limited by too little memory use) Beam search (limit frontier), IDA*, recursive best-first search (RBFS, uses backed-up value),\nand capping memory allotment to search algorithms: SMA*, MA*. Also introduced Bidirectional heuristic search, which has front-to-end and front-to-front variations.\nIntroduces idea of admissibility and consistency (related to triangle equality) of heuristic function. A* is key algorithm,\nparticularly great if it's given a consistent heuristic (optimally efficient). A* prunes nodes that do not lead to optimal solution. Improve A* run-time via satisficing solution (allow\ninadmissible heuristic). \n\n## AIMA Chapter 3.6 Heuristic Functions\nDomination and heuristic function deep dive. Good way of generating a heuristic includes: using a relaxed problem, using a sub-problem (pattern databases and disjoint pattern databases),\nusing landmark points, shortcuts. \n\n## Module 4 Lecture Notes\n Your summary here \n\n## Module 4 Definitions\n* **Admissibility** - Admissible is one that does not overstimate actual cost to reach goal\n* **Consistency** - Cost of current h(n) is less or equal to sum of cost of action + h(n')\n* **Monotonic** - Path cost does not decrease, but may stay same. Strictly monotonic is only increase.\n* **Reference count** - Used in MBS, keeps number of times state has been reached. Used to remove states.\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F3FPMzG9yUls"
      },
      "source": [
        "#Module 5 - Adversarial Search and Games"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iEIOpL7vyWcc"
      },
      "source": [
        "# Russell and Norvig, AIMA chapter 5 \"Adversarial Search\"”\" (5.1-5.3, and 5.5).\n",
        "chapter5_1 = \"\"\" Competitive environments in which there are 2 or more agents give rise to adversarial search problems. Possible stances for viewing multi-agent environments include: as an economy, as part of reason for nondeterministic environment,\n",
        "explicitly model with game-search trees. First takes look at deterministic, fully-observable, 2-play turn-based games where the game is zero-sum. Games include Chess and Go. Move == Action. Ply is a layer. Elements of a game are as follows:\n",
        "State space, initial state, set of actions, transition model, goal state and terminal test, utility function. Similar to chapter 3 (basic search algs). \"\"\"\n",
        "\n",
        "chapter5_2 = \"\"\" begin looking at optimal decisions which includes the introduction of the minimax search algorithm, which involves the backing up (passing information from child to parent.\n",
        "This specifically works for minimax since it is a bottom up algorithm, we get to the leaf level and then start to do the main portion of the algorithm during the unwinding. Also introduces the idea\n",
        "of alliances, being that alliances, regarding of formality, may arise from rational behaviour. Alpha-beta pruning is also introduced, a furthering of the MiniMax algorithm where 2 additional variables\n",
        "are passed and updated appropriately. Alpha is the at least value, while beta is the at most. We can prune a sub-tree when beta is less than alpha. Think specifically of who's making the decision\n",
        "and what they are trying to do when actually pruning the tree. \"\"\"\n",
        "\n",
        "chapter5_3 = \"\"\" Applying eval heuristics to MABS allows us to apply a cutoff test that will greatly reduce computational time. This gives rise to the H-MINIMax alg. Heuristic\n",
        "may be a set of features, each with some material value and assigned weights (weighted linear function). These weighting may be determined by human experience. Applying the heuristic need\n",
        "not only be preprocessing, but in conjunction with the search itself. Main thing is that augmentation should not take too long, as the purpose is to improve speed.\n",
        "This gives rise to Type A and Type B augmentations, the former essentially being depth-limited MiniMax, while type-B involves throwing away bad looking moves. Type-B examples include forward pruning searches such as the beam search, \n",
        "and late move reduction augmented minimax (maintaining a transposition table).\"\"\"\n",
        "\n",
        "chapter5_5 = \"\"\" Stochastic games are ones where there is a random element to the game (ex: Backgammon). Thus, at each decision node,\n",
        "we have Min, Max values in addition to chance (probability of that occuring). Cannot use MiniMax, so use ExpectiMiniMax instead. \n",
        "Difficulty pruning away sub-trees based on likeliness of move occuring (most likely is still <50%), but can often prune via \n",
        "upper-bounding possible utility values. \"\"\"\n",
        "\n",
        "# Russell and Norvig, AIMA Chapter 16 “Making Simple Decisions” (16.1-16.3)\n",
        "chapter16_1 = \"\"\" Further diving into understanding the implications of using probability theory as part of decision-theoretic agents. Recall that probability theory is really what allows us to push\n",
        "beyond only simple task environments. The idea of utility function and expected utility, and that rational agents should maximize expected utility are presented. However, we note that simply rational agents\n",
        "maximizing expected utility is not necessarily all that is needed, though acts as a good guide for guiding agents logically. \"\"\"\n",
        "\n",
        "chapter16_2 = \"\"\" We're introduced to the idea of constraints which we can use to derive MEU and allows us to demonstrate the preferences of a rational agent. The idea of a lottery is also presented,\n",
        "being that our environment is assumed to be? non-deterministic. Thus preference constraints were noted to have the following attributes: orderability, transitivity, continuity, substitutability, monotonicity, and decomposability. \n",
        "These were also called the axioms of utility theory. Note that they relate to the preference of the agent, not the utility function. But we can use these to derive the existence of a utility function \n",
        "though expected utility of a lottery. The absense of a lottery does not contradict this, we simply don't have expected utility as we just need to rank out preferences instead of considering likelihood of occuring. Such\n",
        "utility functions are called value function/ordinal utility functions.\"\"\"\n",
        "\n",
        "chapter16_3 = \"\"\" We discuss determining the actual preferences, through a method called preference elicitation, of humans. Money, risk-aversion, QALY, value of a life are all discussed.\n",
        "Also introduces the optimizer's curse, which essentially is that we overestimate expected utility of the best choice. This leads into a discussion about the difference between normative and descriptive theory and how if the two\n",
        "intersect it would be ideal. However human's are exceptionally irrational at least in relation to normative theory (how they should act). \"\"\"\n",
        "\n",
        "# Lecture notes \n",
        "module5_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module5_definitions = {\n",
        "\"Pruning\" : \"Ignoring irrelevant parts of a tree to improve search time efficiency.\",    \n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "KUc_LgzDvWf-",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "70cb8e0e-77f8-49b5-f035-17c33df435c1"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module5_sections = [\n",
        "  (\"AIMA Chapter 5.1 Game Theory\", chapter5_1),\n",
        "  (\"AIMA Chapter 5.2 Optimal Decisions in Games\", chapter5_2),\n",
        "  (\"AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\", chapter5_3),\n",
        "  (\"AIMA Chapter 5.5 Stochastic Games\", chapter5_5),\n",
        "  (\"AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\", chapter16_1),\n",
        "  (\"AIMA Chapter 16.2 The Basis of Utility Theory\", chapter16_2),\n",
        "  (\"AIMA Chapter 16.3 Utility Functions\", chapter16_3),\n",
        "  (\"Module 5 Lecture Notes\", module5_lecture_notes),\n",
        "]\n",
        "\n",
        "module5_markdown = format_module(5, \"Adversarial Search and Games\", \n",
        "                         module5_sections, \n",
        "                         module5_definitions)\n",
        "display(Markdown(module5_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 5: Adversarial Search and Games\n## AIMA Chapter 5.1 Game Theory\n Competitive environments in which there are 2 or more agents give rise to adversarial search problems. Possible stances for viewing multi-agent environments include: as an economy, as part of reason for nondeterministic environment,\nexplicitly model with game-search trees. First takes look at deterministic, fully-observable, 2-play turn-based games where the game is zero-sum. Games include Chess and Go. Move == Action. Ply is a layer. Elements of a game are as follows:\nState space, initial state, set of actions, transition model, goal state and terminal test, utility function. Similar to chapter 3 (basic search algs). \n\n## AIMA Chapter 5.2 Optimal Decisions in Games\n begin looking at optimal decisions which includes the introduction of the minimax search algorithm, which involves the backing up (passing information from child to parent.\nThis specifically works for minimax since it is a bottom up algorithm, we get to the leaf level and then start to do the main portion of the algorithm during the unwinding. Also introduces the idea\nof alliances, being that alliances, regarding of formality, may arise from rational behaviour. Alpha-beta pruning is also introduced, a furthering of the MiniMax algorithm where 2 additional variables\nare passed and updated appropriately. Alpha is the at least value, while beta is the at most. We can prune a sub-tree when beta is less than alpha. Think specifically of who's making the decision\nand what they are trying to do when actually pruning the tree. \n\n## AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\n Applying eval heuristics to MABS allows us to apply a cutoff test that will greatly reduce computational time. This gives rise to the H-MINIMax alg. Heuristic\nmay be a set of features, each with some material value and assigned weights (weighted linear function). These weighting may be determined by human experience. Applying the heuristic need\nnot only be preprocessing, but in conjunction with the search itself. Main thing is that augmentation should not take too long, as the purpose is to improve speed.\nThis gives rise to Type A and Type B augmentations, the former essentially being depth-limited MiniMax, while type-B involves throwing away bad looking moves. Type-B examples include forward pruning searches such as the beam search, \nand late move reduction augmented minimax (maintaining a transposition table).\n\n## AIMA Chapter 5.5 Stochastic Games\n Stochastic games are ones where there is a random element to the game (ex: Backgammon). Thus, at each decision node,\nwe have Min, Max values in addition to chance (probability of that occuring). Cannot use MiniMax, so use ExpectiMiniMax instead. \nDifficulty pruning away sub-trees based on likeliness of move occuring (most likely is still <50%), but can often prune via \nupper-bounding possible utility values. \n\n## AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\n Further diving into understanding the implications of using probability theory as part of decision-theoretic agents. Recall that probability theory is really what allows us to push\nbeyond only simple task environments. The idea of utility function and expected utility, and that rational agents should maximize expected utility are presented. However, we note that simply rational agents\nmaximizing expected utility is not necessarily all that is needed, though acts as a good guide for guiding agents logically. \n\n## AIMA Chapter 16.2 The Basis of Utility Theory\n We're introduced to the idea of constraints which we can use to derive MEU and allows us to demonstrate the preferences of a rational agent. The idea of a lottery is also presented,\nbeing that our environment is assumed to be? non-deterministic. Thus preference constraints were noted to have the following attributes: orderability, transitivity, continuity, substitutability, monotonicity, and decomposability. \nThese were also called the axioms of utility theory. Note that they relate to the preference of the agent, not the utility function. But we can use these to derive the existence of a utility function \nthough expected utility of a lottery. The absense of a lottery does not contradict this, we simply don't have expected utility as we just need to rank out preferences instead of considering likelihood of occuring. Such\nutility functions are called value function/ordinal utility functions.\n\n## AIMA Chapter 16.3 Utility Functions\n We discuss determining the actual preferences, through a method called preference elicitation, of humans. Money, risk-aversion, QALY, value of a life are all discussed.\nAlso introduces the optimizer's curse, which essentially is that we overestimate expected utility of the best choice. This leads into a discussion about the difference between normative and descriptive theory and how if the two\nintersect it would be ideal. However human's are exceptionally irrational at least in relation to normative theory (how they should act). \n\n## Module 5 Lecture Notes\n Your summary here \n\n## Module 5 Definitions\n* **Pruning** - Ignoring irrelevant parts of a tree to improve search time efficiency.\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qVSQhTVSyYYO"
      },
      "source": [
        "#Module 6 - \"Constraint Satisfaction Problems\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wm4mfW82yaRH"
      },
      "source": [
        "# Russell and Norvig, AIMA, Chapter 6 “Constraint Satisfaction Problems” (6.1-6.5)\n",
        "chapter6_1 = \"\"\" CSP's are interesting as we move away from atomic representations of the TE but towards factored representations. We discuss assignments of variables and consistency with constraints.\n",
        "The goal of CSP is to find a complete and consistent assignment of all variables which means that we have found a solution. Often we can visualize CSP as a constraint graph (everything is a graph...). \n",
        "We also discuss different types of constraints, from simple constraints such as (urary, binary, global) to precendence constraints and disjunctive constraint (the latter 2 often seen in sequential task env.).\n",
        ", to preference constraints. Finally we note the reason why CSP's are good. Problems can be easily formulates as a CSP, solvers are efficient, and effective pruning \"\"\"\n",
        "chapter6_2 = \"\"\" We're introduced to the idea of constraint propogation, which acts as the basis for us reducing the state space.  Local consistency is key to furthering the constraint propogation idea \n",
        "and types of local consistency include node-consistency, arc-consistency, path-consistency, and k-consistency. We're also introduced to AC-3 which uses a queue and evaluates the search space of arcs, eliminating\n",
        "contraidctory arcs. \"\"\"\n",
        "chapter6_3 = \"\"\" We discuss the idea of backtracking in CSP, utilizing the commutativity property of CSP's. Essentially used when a var has multiple variables even after constraint propogation, we being to use\n",
        "searches to find a solution. We discuss backtracking more intelligent through backjumping, which includes simple and conflict-directed backjumping. Both can be used along with constraint learning\n",
        "to improe efficiency of backtracking (and AC-3 as a whole). We also discuss the MRV heuristic which is the idea that inference can be interwoven with search to do forward checking.\n",
        "However forward checking is limited to 1 level, hence the introduction of MAC algorithm which is strictly more powerful than forward checking.\"\"\"\n",
        "chapter6_4 = \"\"\"Tabu search for getting past plateau, constraint weighting to concentrate search on more important constraints. \"\"\"\n",
        "chapter6_5 = \"\"\" Presents idea that the structure of a problem can offer insights on how to find solutions to the problem through breaking down into sub-problems. Specifically, we can attempt to find connected components,\n",
        "solve trees, and utilize the symmetry-breaking constraint depending on the sub-problems. We also are introduced to the idea that constraint graphs can be relatively easily solved if they are a tree, depending on the depth of the tree.\n",
        "Thus we discuss how to reduce a constraint graph into a tree through cycle cutset and tree decomposition.\n",
        "  \"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module6_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module6_definitions = {\n",
        "\"concept\" : \"definition\",    \n",
        "}\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "kKWbCsbfxL3R",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 762
        },
        "outputId": "24dd2521-9840-4346-f062-f8d936239e26"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module6_sections = [\n",
        "  (\"AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\", chapter6_1),\n",
        "  (\"AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\", chapter6_2),\n",
        "  (\"AIMA Chapter 6.3 Backtracking Search for CSPs\", chapter6_3),\n",
        "  (\"AIMA Chapter 6.4 Local Search for CSPs\", chapter6_4),\n",
        "  (\"AIMA Chapter 6.5 The Structure of Problems\", chapter6_5),\n",
        "  (\"Module 6 Lecture Notes\", module6_lecture_notes),\n",
        "]\n",
        "\n",
        "module6_markdown = format_module(6, \"Constraint Satisfaction Problems\", \n",
        "                         module6_sections, \n",
        "                         module6_definitions)\n",
        "display(Markdown(module6_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 6: Constraint Satisfaction Problems\n## AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\n CSP's are interesting as we move away from atomic representations of the TE but towards factored representations. We discuss assignments of variables and consistency with constraints.\nThe goal of CSP is to find a complete and consistent assignment of all variables which means that we have found a solution. Often we can visualize CSP as a constraint graph (everything is a graph...). \nWe also discuss different types of constraints, from simple constraints such as (urary, binary, global) to precendence constraints and disjunctive constraint (the latter 2 often seen in sequential task env.).\n, to preference constraints. Finally we note the reason why CSP's are good. Problems can be easily formulates as a CSP, solvers are efficient, and effective pruning \n\n## AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\n We're introduced to the idea of constraint propogation, which acts as the basis for us reducing the state space.  Local consistency is key to furthering the constraint propogation idea \nand types of local consistency include node-consistency, arc-consistency, path-consistency, and k-consistency. We're also introduced to AC-3 which uses a queue and evaluates the search space of arcs, eliminating\ncontraidctory arcs. \n\n## AIMA Chapter 6.3 Backtracking Search for CSPs\n We discuss the idea of backtracking in CSP, utilizing the commutativity property of CSP's. Essentially used when a var has multiple variables even after constraint propogation, we being to use\nsearches to find a solution. We discuss backtracking more intelligent through backjumping, which includes simple and conflict-directed backjumping. Both can be used along with constraint learning\nto improe efficiency of backtracking (and AC-3 as a whole). We also discuss the MRV heuristic which is the idea that inference can be interwoven with search to do forward checking.\nHowever forward checking is limited to 1 level, hence the introduction of MAC algorithm which is strictly more powerful than forward checking.\n\n## AIMA Chapter 6.4 Local Search for CSPs\nTabu search for getting past plateau, constraint weighting to concentrate search on more important constraints. \n\n## AIMA Chapter 6.5 The Structure of Problems\n Presents idea that the structure of a problem can offer insights on how to find solutions to the problem through breaking down into sub-problems. Specifically, we can attempt to find connected components,\nsolve trees, and utilize the symmetry-breaking constraint depending on the sub-problems. We also are introduced to the idea that constraint graphs can be relatively easily solved if they are a tree, depending on the depth of the tree.\nThus we discuss how to reduce a constraint graph into a tree through cycle cutset and tree decomposition.\n  \n\n## Module 6 Lecture Notes\n Your summary here \n\n## Module 6 Definitions\n* **Concept** - definition\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RYESi2kMybL8"
      },
      "source": [
        "#Module 7 - Logical Agents"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qIRJ4YYCzNHL"
      },
      "source": [
        "# Russell and Norvig, AIMA Chapter 7 “Logical Agents” (Sections 7.1-7.5)\n",
        "\n",
        "chapter7_1 = \"\"\" Your summary here \"\"\"\n",
        "chapter7_2 = \"\"\" Your summary here \"\"\"\n",
        "chapter7_3 = \"\"\" Your summary here \"\"\"\n",
        "chapter7_4 = \"\"\" Your summary here \"\"\"\n",
        "chapter7_5 = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Lecture notes \n",
        "module7_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module7_definitions = {\n",
        "\"concept\" : \"definition\",    \n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "lXNcHuKjyNXa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 541
        },
        "outputId": "a7f986bc-9b85-4f3d-d629-0578a0be7793"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module7_sections = [\n",
        "  (\"AIMA Chapter 7.1 Knowledge-Based Agents\", chapter7_1),\n",
        "  (\"AIMA Chapter 7.2 The Wumpus World\", chapter7_2),\n",
        "  (\"AIMA Chapter 7.3 Logic\", chapter7_3),\n",
        "  (\"AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\", chapter7_4),\n",
        "  (\"AIMA 7.5 Propositional Theorem Proving\", chapter7_5),\n",
        "  (\"Module 7 Lecture Notes\", module7_lecture_notes),\n",
        "]\n",
        "\n",
        "module7_markdown = format_module(7, \"Logical Agents\", \n",
        "                         module7_sections, \n",
        "                         module7_definitions)\n",
        "display(Markdown(module7_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 7: Logical Agents\n## AIMA Chapter 7.1 Knowledge-Based Agents\n Your summary here \n\n## AIMA Chapter 7.2 The Wumpus World\n Your summary here \n\n## AIMA Chapter 7.3 Logic\n Your summary here \n\n## AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\n Your summary here \n\n## AIMA 7.5 Propositional Theorem Proving\n Your summary here \n\n## Module 7 Lecture Notes\n Your summary here \n\n## Module 7 Definitions\n* **Concept** - definition\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vX7lbnJd4sT5"
      },
      "source": [
        "# Your Exam Notes\n",
        "\n",
        "Here are your full set of exam notes that you can use on your midterm.  Choose **Runtime > Run all**, then your notes will be displayed.\n",
        "\n",
        "If you are taking the exam in person, then you must print a copy of your notes for yourself.   Here's how to print:\n",
        "* Run the cell below. \n",
        "* Click its \"Mirror cell in tab\" button (this is located in the upper right corner of the cell, immediately to the right of the trash can).  This will open the cell.  \n",
        "* In Chrome, you can then print your notes using the **File > Print** option from the browser's menu.  \n",
        "* In the print options, change the layout from Landscape to Portrait. \n",
        "* Clicking the Save, will save a PDF that you can then print.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "T3cm1WV542_a",
        "outputId": "64802877-8488-46d1-a7c5-6230fed62558"
      },
      "source": [
        "# This will generate the full set of notes that you can use for your exam.\n",
        "full_markdown = \"\\n\\n\".join([module1_markdown, \n",
        "                             module2_markdown, \n",
        "                             module3_markdown, \n",
        "                             module4_markdown, \n",
        "                             module5_markdown,\n",
        "                             module6_markdown,\n",
        "                             module7_markdown,]\n",
        "                       )\n",
        "display(Markdown(full_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 1: Rational Agents and Task Environments\n## AIMA Chapter 1.1 What is AI?\nA brief overview of the field of A.I. and the key terminology underlying the study of A.I. including rationality, thinking, logic, and intelligence. Proposed the Turing test. Composed of NLP (communicate like human), Knowledge storing and representation (from sensors), automated reasoning (drawing conclusions), and Machine Learning (adapting\nto new circumstance and extracting patterns) and extended Turing Test (How entity perceives, aka computer vision, and manipulation of environment, aka robotics/actuators). \nThe idea of what it means to think humanly and logic itself were presented, the latter begining with Syllogisms and needing a complete/total knowledge base (which is impractical).\nNow logic encompasses probability theory, allowing the making of decisions even when the world is not completely observable or uncertainty exists (nondeterministic, stochastic task environments)\n\n## AIMA Chapter 27.1 The Limits of AI\n Limits of AI and initial arguments against AI including an argument from informality, that there are too many informal rules in reality\nand too difficult to quanitifed/represented in a knowledge base. This is sort of true, giving rise to the qualification problem, probability theory is supposedly a solution. Essentially, AI does not offer complete answers and therefore is impossible.\nAnother argument is from disability. The only understanding is that programs can't be exactly human (because they are not). Mathematical objection is another claim against AI. Argument is that \nAI is limited by Godel's incompleteness theorem, being that machinese are formal systems while humans are not. However, incompleteness only applies to math, not computers which are finite (just like people). Computers != turing machines\n\n## AIMA Chapter 27.1 Can Machines Really Think?\n Your summary here \n\n## AIMA 2.1 Agents and Environments\n Deep dive into what makes an agent intelligent. Agent is intelligent if it perceives env. through sensors and acts through its actuators. The actions it takes is limited to percept sequence as agent function maps percept sequence to actions.\nThus, an agent cannot do something that it does not know is possble (not in its set of actions). Also discusses the agent function and agent program, the prior being a external mathematical abstraction while the latter is a internal implementation of the former.\n \n\n## AIMA 2.2 Good Behavior: The Concept of Rationality\n Focus on the idea of rationality and what is good behavior. Specifically, that pure consequentialism (evaluating behaviour purely on outcome) is insufficient. Introduced the notion of a performance measure, which we later know to be\na key component of the PEAS-agent building framework. Rational decision-making requires 4 components: the perfomrnace measure, prior knowledge of the task env, set of actions that can be performed, and percept sequence up until the decision point.\nThis gives rise to the definition of a rational agent, an agent which selects and performs the action that is expected to achieve best possible outcome of a prespecified performance measure for each\npossible percept sequence that is built off of previous percept sequence (to date) and built in knowledge/learning.\n\n\n## AIMA 2.3 The Nature of Environments\n Discussion of the task environment, which is where the problem is. Well-designing agents often rely on description of task environment amap. Environments can be partially or fully observable, and observability\nis discussed in relation to how much of the environment the agent's sensors can perceive. An unobservable environment is one in which an agent has no sensors (or relevant sensors).\nTypes of environments: Single/multi-agent, deterministic/non-deterministic/stochastic, episodic/sequential, static/dynamic, discrete/continuous, (known/unknown).\n\nDeterministic environments are one where the environment is only impacted by the actions of the agent. Non-deterministic and stochastic both relate to uncertainty, though stochastic adds probability to outcome. \nNote that knowing != observability. I can know the environment but have partial observability (think Solitaire).\nMost difficult environments are those like reality, multi-agent, non-deterministic/stochastic, sequential, dynamic, continuous, and unknown.\n \n\n## AIMA 2.4 The Structure of Agents\n Desigining and structuring agents - think PEAS. The types of agents are simple reflex, model-based, goal-based, and utility-based. There are also extension types being model-free and learning agent.\nThe learning agent involves a loop involving the critic (evaluate performance in relation to PM), learning element, problem, generator, and performance element. Also discusses the different representations of the world being\nAtomic (think basic search algorithms, informed/uninformed ones), Factored (think CSP algs inc. AC3), and Structured. Physical memory can also be represented in a localist and distributed manner.\n\n## Module 1 Lecture Notes\n PEAS for agent design: Performance Measure, (task) environment, actuators, sensors \n\n## Module 1 Definitions\n* **Agent** - Anything that perceives its environment through sensors and interacts with environment through actuators\n* **Percept** - content received by sensors\n* **Percept sequence** - Entire history of percept that an agent has received\n* **Agent function** - Mathematical abstraction to map percept sequence to set of actions, external\n* **Agent program** - Internal implementation of agent function\n* **Intelligence** - How one thinks and acts\n* **A.i.** - The understanding and building of intelligent entities\n* **Rationality** - Doing the right thing, given the objective provided/specified. Rationality is reliant on action, not only thought.\n* **Introspection** -  Ability to catch one's own thoughts\n* **Psychological experiments** - Having thoughts that are observable through experimentation\n* **Brain imaging** -  Physically being able to watch a brain (not necessarily biological) in action\n* **Rational agent** - Agent that acts to achieve best possible (rational) or best expected outcome (limited rationality)\n* **Standard model** - Doing the right thing based on the predefined/specified objective, ex: Max or Min value\n* **Value alignment problem** - Simply achieving objective may not be desirable, the problem of aligning objective to true preferences\n* **Omniscience** - Knowing the eventual outcome and acting accordingly, there is no uncertainty. Essentially allows for perfection which != rationality.\n* **Information gathering** - Doing actions in order to augment future percepts\n* **Autonomy** - Agent being able to learn based on own percepts and learning processes independent from creator/designed.\n* **Limited rationality** - When agent achieves the best expected value\n* **Weak ai** - Machines acting as if intelligent, but not thinking curiously\n* **Strong ai** - Consciously thinking machines\n\n\n# Module 2: Python\n## Module 2 Lecture Notes\n Your summary here \n\n## Module 2 Definitions\n* **Mutability** - your definition here\n\n\n# Module 3: Search Problems\n## AIMA Chapter 3.1 Problem-Solving Agents\n Problem Solving Agents. Process of goal solving is goal formulation (destination state), problem formation, search, and execution. World is represented atomically, and search algorithms are either informed or uninformed.\nSearch problems consist of 6 components: Initial state, a state space, goal state, actions, transition model, and action cost function. \n \n\n## AIMA Chapter 3.2 Example Problems\n Discusses standardized problems, which are hypothetical problems/environments, and real-world problems. Example of standardized problem is grid world, real-world problem is route-finding. \n\n## AIMA Chapter 3.3 Search Algorithms\n Introduces algorithms include: Best first search (choose node with minimum eval function value f(n). Best first search uses a priority queue. Graph search is if a search algorithm checks for redundant paths, otherwise algorithm is a\ntree-like search. Completeness is dependent on the systematic way an algorithm works. Problems may arise in infinite search spaces. d = depth, m = max actions in any path, b is branching factor (number of successors per node\n\n## AIMA Chapter 3.4 Uninformed Search Strategies\n Overview of uninformed search algorithms. BFS (queue), Dijkstra's/UCS (exact opposite of what name suggests), DFS (stack, memory constraint, not complete or optimal), DLS (same problem as DFS, but limit depth),\nIDDFS (mix of BFS and DFS, O(b^d) or, O(b^m) if no solution, does not store previous states in memory, just regenerates them), Bidirectional search (search from initial to goal and goal to initial concurrently, 2 frontiers, 2 reached states, solution when frontier meet.\n\n\n## Module 3 Lecture Notes\n We deal the most basic of environments for basic search algorithms: Epsidoci, fully-observable, discrete, static, deterministic, and known. If environment is unknown, execute action at random.\n\n## Module 3 Definitions\n* **Problem-solving agent** - an agent that can formulate a plan via search\n* **Uninformed** -  No estimate of how far agent is from goal state.\n* **Solution** - Path from the initial state to goal state that is optimal if is lowest-path cost\n* **Completeness** - Answer will be found if it exists, and failure if not\n* **Optimality** - Solution will be of lowest path cost\n\n\n# Module 4: Informed Search\n## AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\n Informed Heuristic Search strategies: Greedy best-first search, speedy search, A*, weighted A*, (search limited by too little memory use) Beam search (limit frontier), IDA*, recursive best-first search (RBFS, uses backed-up value),\nand capping memory allotment to search algorithms: SMA*, MA*. Also introduced Bidirectional heuristic search, which has front-to-end and front-to-front variations.\nIntroduces idea of admissibility and consistency (related to triangle equality) of heuristic function. A* is key algorithm,\nparticularly great if it's given a consistent heuristic (optimally efficient). A* prunes nodes that do not lead to optimal solution. Improve A* run-time via satisficing solution (allow\ninadmissible heuristic). \n\n## AIMA Chapter 3.6 Heuristic Functions\nDomination and heuristic function deep dive. Good way of generating a heuristic includes: using a relaxed problem, using a sub-problem (pattern databases and disjoint pattern databases),\nusing landmark points, shortcuts. \n\n## Module 4 Lecture Notes\n Your summary here \n\n## Module 4 Definitions\n* **Admissibility** - Admissible is one that does not overstimate actual cost to reach goal\n* **Consistency** - Cost of current h(n) is less or equal to sum of cost of action + h(n')\n* **Monotonic** - Path cost does not decrease, but may stay same. Strictly monotonic is only increase.\n* **Reference count** - Used in MBS, keeps number of times state has been reached. Used to remove states.\n\n\n# Module 5: Adversarial Search and Games\n## AIMA Chapter 5.1 Game Theory\n Competitive environments in which there are 2 or more agents give rise to adversarial search problems. Possible stances for viewing multi-agent environments include: as an economy, as part of reason for nondeterministic environment,\nexplicitly model with game-search trees. First takes look at deterministic, fully-observable, 2-play turn-based games where the game is zero-sum. Games include Chess and Go. Move == Action. Ply is a layer. Elements of a game are as follows:\nState space, initial state, set of actions, transition model, goal state and terminal test, utility function. Similar to chapter 3 (basic search algs). \n\n## AIMA Chapter 5.2 Optimal Decisions in Games\n begin looking at optimal decisions which includes the introduction of the minimax search algorithm, which involves the backing up (passing information from child to parent.\nThis specifically works for minimax since it is a bottom up algorithm, we get to the leaf level and then start to do the main portion of the algorithm during the unwinding. Also introduces the idea\nof alliances, being that alliances, regarding of formality, may arise from rational behaviour. Alpha-beta pruning is also introduced, a furthering of the MiniMax algorithm where 2 additional variables\nare passed and updated appropriately. Alpha is the at least value, while beta is the at most. We can prune a sub-tree when beta is less than alpha. Think specifically of who's making the decision\nand what they are trying to do when actually pruning the tree. \n\n## AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\n Applying eval heuristics to MABS allows us to apply a cutoff test that will greatly reduce computational time. This gives rise to the H-MINIMax alg. Heuristic\nmay be a set of features, each with some material value and assigned weights (weighted linear function). These weighting may be determined by human experience. Applying the heuristic need\nnot only be preprocessing, but in conjunction with the search itself. Main thing is that augmentation should not take too long, as the purpose is to improve speed.\nThis gives rise to Type A and Type B augmentations, the former essentially being depth-limited MiniMax, while type-B involves throwing away bad looking moves. Type-B examples include forward pruning searches such as the beam search, \nand late move reduction augmented minimax (maintaining a transposition table).\n\n## AIMA Chapter 5.5 Stochastic Games\n Stochastic games are ones where there is a random element to the game (ex: Backgammon). Thus, at each decision node,\nwe have Min, Max values in addition to chance (probability of that occuring). Cannot use MiniMax, so use ExpectiMiniMax instead. \nDifficulty pruning away sub-trees based on likeliness of move occuring (most likely is still <50%), but can often prune via \nupper-bounding possible utility values. \n\n## AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\n Further diving into understanding the implications of using probability theory as part of decision-theoretic agents. Recall that probability theory is really what allows us to push\nbeyond only simple task environments. The idea of utility function and expected utility, and that rational agents should maximize expected utility are presented. However, we note that simply rational agents\nmaximizing expected utility is not necessarily all that is needed, though acts as a good guide for guiding agents logically. \n\n## AIMA Chapter 16.2 The Basis of Utility Theory\n We're introduced to the idea of constraints which we can use to derive MEU and allows us to demonstrate the preferences of a rational agent. The idea of a lottery is also presented,\nbeing that our environment is assumed to be? non-deterministic. Thus preference constraints were noted to have the following attributes: orderability, transitivity, continuity, substitutability, monotonicity, and decomposability. \nThese were also called the axioms of utility theory. Note that they relate to the preference of the agent, not the utility function. But we can use these to derive the existence of a utility function \nthough expected utility of a lottery. The absense of a lottery does not contradict this, we simply don't have expected utility as we just need to rank out preferences instead of considering likelihood of occuring. Such\nutility functions are called value function/ordinal utility functions.\n\n## AIMA Chapter 16.3 Utility Functions\n We discuss determining the actual preferences, through a method called preference elicitation, of humans. Money, risk-aversion, QALY, value of a life are all discussed.\nAlso introduces the optimizer's curse, which essentially is that we overestimate expected utility of the best choice. This leads into a discussion about the difference between normative and descriptive theory and how if the two\nintersect it would be ideal. However human's are exceptionally irrational at least in relation to normative theory (how they should act). \n\n## Module 5 Lecture Notes\n Your summary here \n\n## Module 5 Definitions\n* **Pruning** - Ignoring irrelevant parts of a tree to improve search time efficiency.\n\n\n# Module 6: Constraint Satisfaction Problems\n## AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\n CSP's are interesting as we move away from atomic representations of the TE but towards factored representations. We discuss assignments of variables and consistency with constraints.\nThe goal of CSP is to find a complete and consistent assignment of all variables which means that we have found a solution. Often we can visualize CSP as a constraint graph (everything is a graph...). \nWe also discuss different types of constraints, from simple constraints such as (urary, binary, global) to precendence constraints and disjunctive constraint (the latter 2 often seen in sequential task env.).\n, to preference constraints. Finally we note the reason why CSP's are good. Problems can be easily formulates as a CSP, solvers are efficient, and effective pruning \n\n## AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\n We're introduced to the idea of constraint propogation, which acts as the basis for us reducing the state space.  Local consistency is key to furthering the constraint propogation idea \nand types of local consistency include node-consistency, arc-consistency, path-consistency, and k-consistency. We're also introduced to AC-3 which uses a queue and evaluates the search space of arcs, eliminating\ncontraidctory arcs. \n\n## AIMA Chapter 6.3 Backtracking Search for CSPs\n We discuss the idea of backtracking in CSP, utilizing the commutativity property of CSP's. Essentially used when a var has multiple variables even after constraint propogation, we being to use\nsearches to find a solution. We discuss backtracking more intelligent through backjumping, which includes simple and conflict-directed backjumping. Both can be used along with constraint learning\nto improe efficiency of backtracking (and AC-3 as a whole). We also discuss the MRV heuristic which is the idea that inference can be interwoven with search to do forward checking.\nHowever forward checking is limited to 1 level, hence the introduction of MAC algorithm which is strictly more powerful than forward checking.\n\n## AIMA Chapter 6.4 Local Search for CSPs\nTabu search for getting past plateau, constraint weighting to concentrate search on more important constraints. \n\n## AIMA Chapter 6.5 The Structure of Problems\n Presents idea that the structure of a problem can offer insights on how to find solutions to the problem through breaking down into sub-problems. Specifically, we can attempt to find connected components,\nsolve trees, and utilize the symmetry-breaking constraint depending on the sub-problems. We also are introduced to the idea that constraint graphs can be relatively easily solved if they are a tree, depending on the depth of the tree.\nThus we discuss how to reduce a constraint graph into a tree through cycle cutset and tree decomposition.\n  \n\n## Module 6 Lecture Notes\n Your summary here \n\n## Module 6 Definitions\n* **Concept** - definition\n\n\n# Module 7: Logical Agents\n## AIMA Chapter 7.1 Knowledge-Based Agents\n Your summary here \n\n## AIMA Chapter 7.2 The Wumpus World\n Your summary here \n\n## AIMA Chapter 7.3 Logic\n Your summary here \n\n## AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\n Your summary here \n\n## AIMA 7.5 Propositional Theorem Proving\n Your summary here \n\n## Module 7 Lecture Notes\n Your summary here \n\n## Module 7 Definitions\n* **Concept** - definition\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    }
  ]
}