{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of FINAL Template_for CIS521_midterm notes.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "brNDGX-kvzds"
      },
      "source": [
        "# Template for Notes for CIS 521 Midterm 1\n",
        "\n",
        "The midterm exam for CIS 521 will be **open notes**. No other matierals are allowed for the exam - you may not use the textbook, the internet, a Python envionement, or communicate with anyone during the exam. \n",
        "\n",
        "This python notebook is where you should save all of your notes.  It provides a structured template that allows you to add summaries for each of the textbook sections that are part of the required reading, plus room for lecture notes and definitions.\n",
        "\n",
        "Your summaries must be written in your own words.  Do not copy and paste from the textbook.  Do not share your notes with classmates, or copy any of your classmates' notes.  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aPtM8g7R0Pds"
      },
      "source": [
        "# Instructions\n",
        "\n",
        "1. Make a copy of this Google Colab file. From the colab menu, pick **File > Save a copy in Drive**.  That will save a copy of this notebook in your Google drive.\n",
        "2. Go through each section of the notebook, and fill in all the summaries for each chapter and its corresponding sections.  Be sure to save your work frequently!\n",
        "3. Read the notes below about how you can format your summaries.\n",
        "4. In addition to these notes, you are allowed 1 page (both sides) for figures or handwritten notes.\n",
        "5. Submit your version of this notebook to Gradescope before you start the exam.\n",
        "6. If you are taking the exam in person, then you must print a copy of your notes for yourself.   Here's how to print: \n",
        "* Run the last cell of this notebook\n",
        "* Click the \"Mirror cell in tab\" button (this is located in the upper right corner of the cell, immediately to the right of the trash can).  This will open the cell.  \n",
        "* In Chrome, you can then print your notes using the **File > Print** option from the browser's menu.  \n",
        "* In the print options, change the layout from Landscape to Portrait. \n",
        "* Clicking the Save, will save a PDF that you can then print.\n",
        "\n",
        "## Formatting Notes\n",
        "\n",
        "1. The max length for each section's summary is 500 words / 3000 characters.\n",
        "2. You can also write a 2000 word summary of the lectures for each module.\n",
        "3. You may also write definitions of concepts for each module.  The definitions should be written in your own words (not copy and pasted), and they should be no more than 20 words long for each concept. \n",
        "2. You can use Markdown formatting for your summaries. \n",
        "3. You can use Latex formatting for equations. Put two dollar signs before and after your equation like this: \n",
        "$$ \\epsilon = \\frac{1}{mc^2} $$\n",
        "4. When you write python code or psuedocode algorithms, you should surround it with triple backticks like this:\n",
        "```\n",
        "def function():\n",
        "      print('hello world')\n",
        "```\n",
        "5. Write \\ whenever you want to keep writing on the next line without a newline being inserted."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cw5IpDLCZa4W"
      },
      "source": [
        "## Example summary\n",
        "\n",
        "This is a sample summary of [Section 2.1.1](https://web.stanford.edu/~jurafsky/slp3/2.pdf ) of Jurfasky and Martin's Speech and Language Processing's book\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lzaavW96tcxM",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f0242208-444f-4bac-9911-67fb605f74e2"
      },
      "source": [
        "chapter_2_1_example = \"\"\"\n",
        "Regular expressions are an algebraic notation for characterizing \\\n",
        "a set of strings that are useful for searching for a pattern or through a corpus.\\\n",
        "Only the first matching result will be shown. The search string is case sensitive, \\\n",
        "but it can consist of a single character or a sequence of characters. \\\n",
        "While square braces specify a disjunction of characters, the dash can more easily look \\\n",
        "for characters over a range. The caret specifies the non-existence of a character. \\\n",
        "Additionally, the (?) is used to for flexibility on a single character, meaning \\\n",
        "\"the preceding character or nothing\". More importantly, the Kleene star means zero or \\\n",
        "more occurrences of the immediately previous character or regular expression in square braces.\\\n",
        "...\n",
        "\"\"\"\n",
        "\n",
        "length = len(chapter_2_1_example.split(\" \"))\n",
        "print(\"The example summary is {len} words long, so you could write an additional \\\n",
        "{remaining} words.\".format(len=length, remaining=(500-length)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The example summary is 116 words long, so you could write an additional 384 words.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s3Uk9gQTevcj"
      },
      "source": [
        "## Example summary length\n",
        "\n",
        "The maximum summary length for each section is 500 words / 3000 characters.  Hopefully you'll find that to be quite generous. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8DvZ57A-djo1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9ee31d5f-d0a3-4840-fb5f-f00fc4b3bf81"
      },
      "source": [
        "# This is an example of a 500 word placeholder\n",
        "placeholder = \"\"\"ultrices vitae auctor eu augue ut lectus arcu bibendum at varius \\\n",
        " vel pharetra vel turpis nunc eget lorem dolor sed viverra ipsum nunc aliquet \\\n",
        " bibendum enim facilisis gravida neque convallis a cras semper auctor neque vitae \\\n",
        " tempus quam pellentesque nec nam aliquam sem et tortor consequat id porta nibh \\\n",
        " venenatis cras sed felis eget velit aliquet sagittis id consectetur purus ut \\\n",
        " faucibus pulvinar elementum integer enim neque volutpat ac tincidunt vitae semper \\ \n",
        " quis lectus nulla at volutpat diam ut venenatis tellus in metus vulputate eu \\\n",
        " scelerisque felis imperdiet proin fermentum leo vel orci porta non pulvinar neque \\ \n",
        " laoreet suspendisse interdum consectetur libero id faucibus nisl tincidunt eget \\\n",
        " nullam non nisi est sit amet facilisis magna etiam tempor orci eu lobortis \\\n",
        " elementum nibh tellus molestie nunc non blandit massa enim nec dui nunc mattis \\\n",
        " enim ut tellus elementum sagittis vitae et leo duis ut diam quam nulla porttitor \\\n",
        " massa id neque aliquam vestibulum morbi blandit cursus risus at ultrices mi tempus \\\n",
        " imperdiet nulla malesuada pellentesque elit eget gravida cum sociis natoque \\\n",
        " penatibus et magnis dis parturient montes nascetur ridiculus mus mauris vitae \\\n",
        " ultricies leo integer malesuada nunc vel risus commodo viverra maecenas accumsan \\\n",
        " lacus vel facilisis volutpat est velit egestas dui id ornare arcu odio ut sem nulla \\\n",
        " pharetra diam sit amet nisl suscipit adipiscing bibendum est ultricies integer \\\n",
        " quis auctor elit sed vulputate mi sit amet mauris commodo quis imperdiet massa \\\n",
        " tincidunt nunc pulvinar sapien et ligula ullamcorper malesuada proin libero nunc \\\n",
        " consequat interdum varius sit amet mattis vulputate enim nulla aliquet porttitor \\\n",
        " lacus luctus accumsan tortor posuere ac ut consequat semper viverra nam libero \\\n",
        " justo laoreet sit amet cursus sit amet dictum sit amet justo donec enim diam \\\n",
        " vulputate ut pharetra sit amet aliquam id diam maecenas ultricies mi eget mauris \\\n",
        " pharetra et ultrices neque ornare aenean euismod elementum nisi quis eleifend quam \\\n",
        " adipiscing vitae proin sagittis nisl rhoncus mattis rhoncus urna neque viverra justo \\\n",
        " nec ultrices dui sapien eget mi proin sed libero enim sed faucibus turpis in \\\n",
        " eu mi bibendum neque egestas congue quisque egestas diam in arcu cursus euismod \\\n",
        " quis viverra nibh cras pulvinar mattis nunc sed blandit libero volutpat sed cras \\\n",
        " ornare arcu dui vivamus arcu felis bibendum ut tristique et egestas quis ipsum \\\n",
        " suspendisse ultrices gravida dictum fusce ut placerat orci nulla pellentesque \\\n",
        " dignissim enim sit amet venenatis urna cursus eget nunc scelerisque viverra mauris \\\n",
        " in aliquam sem fringilla ut morbi tincidunt augue interdum velit euismod in \\\n",
        " pellentesque massa placerat duis ultricies lacus sed turpis tincidunt id aliquet \\\n",
        " risus feugiat in ante metus dictum at tempor commodo ullamcorper a lacus vestibulum \\\n",
        " sed arcu non odio euismod lacinia at quis risus sed vulputate odio ut enim blandit \\\n",
        " volutpat maecenas volutpat blandit aliquam etiam erat velit scelerisque in \\\n",
        " dictum non consectetur a erat nam at lectus urna\"\"\"\n",
        "\n",
        "print(\"The length in words is\", len(placeholder.split(' ')))\n",
        "print(\"The length in characters is\", len(placeholder))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 500\n",
            "The length in characters is 3030\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5yG3-YncJfTB"
      },
      "source": [
        "# Permissions to use your summaries\n",
        "\n",
        "We'd like to use your summaries to help develop better educational resources like [remember.school](https://remember.school) or a question-answering system about the textbook.  \n",
        "\n",
        "1. Please enter your name.  \n",
        "\n",
        "2. If you grant permission for us to use your summaries under a [Creative Commons License](https://creativecommons.org/licenses/by/4.0/), please set ```permission = True```.  If you do not grant permission, please set it to false. Your answer won't affect your grade.\n",
        "\n",
        "3. If you grant us permission, and you'd like to remain anonymous, set ```anonymous = True```.  We'll remove your name before sharing your summaries with anyone. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2RECnJArJew4"
      },
      "source": [
        "name = \"\"\n",
        "permission = True\n",
        "anonymous = False"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GZZXyrV-xuAp"
      },
      "source": [
        "# Module 1 - Intro to AI"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y2NfE1CKxzS-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "85b27fa3-ec20-4fc1-cec4-5c05c988e487"
      },
      "source": [
        "# Russell and Norvig, AIMA 4th Edition, Chapter 1 “Introduction”, Section 1.1 “What is AI”\n",
        "\n",
        "chapter1_1 = \"\"\" There is some debate over the meaning of being intelligent, leading to the following four approaches:\n",
        "<table>\n",
        "<tr><td></td><td><b>Thinking</b></td><td><b>Acting</b></td></tr>\n",
        "<tr>\n",
        "<td><b>Humanly</b></td>\n",
        "<td>We can model how humans think in three main ways:\n",
        "<ul><li><b>introspection</b> - monitoring our own thoughts</li>\n",
        "<li><b>psychological experiments</b> - observing people in action</li>\n",
        "<li><b>brain imaging</b> - observing the brain in action</li></ul>\n",
        "These developments have led to fields like <b>cognitive science</b> which use experimental techniques from psychology, and some of these brain models have made their way into\n",
        "computer vision research.</td>\n",
        "<td>The <b>Turing test</b> asks if a machine can appear human to a human interrogator. In order to do this, a Turing machine would need the following abilities:\n",
        "<ul><li><b>natural language processing</b> to be able to communicate the way a human does</li>\n",
        "<li><b>knowledge representation</b> to store and reason about what it perceives</li>\n",
        "<li><b>automated reasoning</b> to respond to questions and come up with new conclusions</li>\n",
        "<li><b>machine learning</b> to extrapolate patterns to apply to new situations</ul>\n",
        "To apss the more complicated <b>total Turing test</b> which requires interacting with people, it would also need:\n",
        "<ul><li><b>computer vision</b> to perceive objects in the real world</li>\n",
        "<li><b>robotics</b> to move and be able to manipulate objects</li></ul></td></tr>\n",
        "<tr>\n",
        "<td><b>Rationally</b></td>\n",
        "<td>The \"Laws of Thought\" approach is modeled after attempts to codify \"right thinking\" by philosophers like Aristotle. He defined certain <b>syllogisms</b> for\n",
        "logical reasoning that formed the basis for the field of <b>logic</b>. In efforts since these are codified into a formal language via the <b>logicist</b> tradition.\n",
        "When outcomes cannot be defined with <i>certainty</i>, the theory of <b>probability</b> allows rigorous reasoning with information that is uncertain.</td>\n",
        "<td>An <b>agent</b> is something that acts. Computer argents are expected to operate autonomously by perceiving its\n",
        "environment, persisting over an extended time period, adapting to change, and identifying and pursuing goals. A <b>rational agent</b> is one\n",
        "that acts in a way that achieves the best expected outcome when given an uncertain situation.This has the advantage that it is more general than the\n",
        "\"laws of thought\" approach as there are several ways to achieve rationality beyond inference and it is amenable to the scientific process as it can\n",
        "be rigorously defined and generalized. Specific agent designs and goals are less ambiguous to define than mimicking human behavior.</td></tr>\n",
        "</table><br />\n",
        "As AI becomes more complicated, the specific goals and objectives are more ambiguous to define. How should a self-driving car\n",
        "behave to avoid endangering other errant drivers, passenger comfort, etc? These lead to the <b>value alignment problem</b> where we ask if the values\n",
        "in the AI match those of humans.\"\"\" \n",
        "\n",
        "# Russell and Norvig, AIMA 4th Edition, Chapter 27 “Philosophical Foundations”, Sections 27.1 “The Limits of AI” and 27.2 “Can Machines Really Think?”\n",
        "chapter27_1 = \"\"\"John Searle, a philosopher, proposed the concept of <b>strong AI</b> and <b>weak AI</b>, where the former denotes machines that are actually\n",
        "capable of human thought and the altter referring to machines that merely act like humans. The notion of weak AI has since been disputed and indeed, Alan Turing anticipated\n",
        "several of the main arguments that have been made against AI:\n",
        "<ol>\n",
        "<li>Turing's \"argument from the informality of behavior\" stated that humans do not have a formal set of rules and instead use informal guidance to make decisions. This view\n",
        "was further argued for by Dreyfus who wrote books on what AI couldn't do, but these models have since been discarded as the idea of <b>Good Old-Fashioned AI (GOFAI)</b>.\n",
        "Indeed, while the <b>qualification problem</b> has shown that some problems are difficult to explicitly model, deep learning and other techniques have proven to be able\n",
        "to structure informal situations. Another variant of this argument is <b>embodied cognition</b>: cognition takes place throughout the whole body, placed in an environment.\n",
        "Then, we only need to study the agent's behavior as a whole - under such a view, robotics, vision, etc. become central as opposed to peripheral.</li>\n",
        "<li>The \"argument from disability\" states that a machine can never do certain things - Turing listed \"tell right from wrong, make mistakes, fall in love, enjoy strawberries\n",
        "and cream, make someone fall in love with it, learn from experience, etc.\" Computers have shown to be able to do many of these things, including making significant\n",
        "contributions to fields like chemistry, computer science, even creating new art via style transfer.</li>\n",
        "<li>Gobel stated that for any axiomatic framework <i>F</i> that is capable of doing arithmetic, one can construct\n",
        "a Godel sentence <i>G(F)</i> which ahd two properties:\n",
        "<ul><li><i>G(F)</i> is a sentence of <i>F</i> that cannot be proved within <i>F</i>\n",
        "<li>if <i>F</i> is consistent, then it follows that <i>G(F)</i> is true.</li></ul>\n",
        "It follows from this that by the incompleteness theorem, machines are limited and cannot establish their own Godel sentence. However, this has been contended with for a couple\n",
        "of reasons:\n",
        "<ol><li>Just because one agent, e.g. human cannot assert something, that does not mean other people or machines can</li>\n",
        "<li>Godel's incompleteness theorem apply to <i>mathematics</i> and not necessarily <i>computers</i> as no one is capable of proving something that is impossible to prove.\n",
        "Humans sometimes assume their own consistency only to be proven false later, as in the four-color map problem.</li>\n",
        "<li>This fails to consider finite limitations for computers and human brains, who can be described as very large propositional logic systems. But just as a human can change\n",
        "its decision, a computer may have different results with machine learning or software rewriting.</li></ol>\n",
        "</ol>\n",
        "The <b>turing test</b> has not been of much focus in the AI community although some Chatbots have shown a lot of potential.\n",
        "\"\"\"\n",
        "chapter27_2 = \"\"\"Some further debate has gone into whether it is even important that machines think - for example, submarines don't necessarily swim and airplanes don't\n",
        "necessarily fly. Turing himself suggested the notion of <b>polite convention</b> - that is, we take what other people say at face value and don't try to break apart their\n",
        "inner thoughts or how they reason about what they say, so similar leniencey should be given to machines. There have been some philosophers who disagree with this - Searle\n",
        "also came up with a thoguht experiment called the <b>Chinese room</b>, in which a translator looks up every single English word that comes in and presses a button resulting\n",
        "in a correct Chinese translation on the other end. Searle argues that the translator does not in fact know Chinese even though the end user would be getting perfect\n",
        "Chinese translations. Searle was thus a big proponent of <b>biological naturalism</b> which states that mental states are caused by low-level biological processes in the\n",
        "neurons themselves and the intrinsic properties of these are what define intelligence. Thus much of this debate surrounds the concept of <b>consciousness</b> or awareness\n",
        "of the outside world, and the subjective experience of living, e.g. the concept of self. The <b>qualia</b> or intrinsic nature of these experiences is not so important\n",
        "according to Turing, and further some of these aspects can be programmed into machines but this <i>awareness</i> is still mostly a part of the human experience.\"\"\"\n",
        "\n",
        "# Russell and Norvig, AIMA 4th Edition, Chapter 2 “Intelligent Agents” (whole chapter)\n",
        "chapter2_1 = \"\"\"Here, an <b>agent</b> is any object that is capable of perceiving its <b>environment</b> with the use of <b>sensors</b> and then acting on these <b>percepts</b>\n",
        "using <b>actuators</b>. Some examples of agents include a piece of software which receives packets and I/O from a user and then sends packets or displays information in\n",
        "response, or a robot that perceives through its camera lens and infrared range finders and then moves itself based on the environment it perceives. In general, an agent\n",
        "considers its entire <b>percept sequence</b> and the agent's choice of what action to take at any given instant depends on any built-in knowledge provided and on the\n",
        "percept sequence up to the current moment, and not on anything it has yet to perceive. The way that an agent behaves can be described by its <b>agent function</b> which maps\n",
        "a percept sequence to a specific action.\n",
        "<br /><br />\n",
        "One interpretation of the agent function is that it <i>tabulates</i>the agent function - but such a table could be infinitely large and this is an <i>external</i> representation\n",
        "of the agent's choices. <i>Internally</i>, the agent function is implemented by the <b>agent program</b>. The key difference between the two is that the agent function is an\n",
        "abstract mathematical function while the agent program is its actual implementation.\n",
        "<br /><br />\n",
        "A final note is that the idea of an agent is meant to be a paradigm or tool for analyzing a system as opposed to an absolute characterization of the system. There is no\n",
        "inherent dichotomy between agents and non-agents in the real world. In some sense, most of engineering involves artifacts that interact the world and artifical intelligence\n",
        "operates at the end of the spectrum where resources have significant computational resources and are tasked with making nontrivial decisions.\"\"\"\n",
        "chapter2_2 = \"\"\"A <b>rational agent</b> can be defined as one that chooses the right action. But in terms of what defines \"right\", AI tends to stick to the notion of\n",
        "<b>consequentialism</b>, or evaluating its success by the consequences of its actions. The <b>performance measure</b> is a function that evaluates any given sequence of successive\n",
        "environment states. Since machines do not have desires or preferences, at least not initially, it is better in general to design performance measures based on the outcome one wants to achieve in an environment (the goal state) as opposed to how one thinks \n",
        "that an agent <i>should</i> behave.\n",
        "<br /><br />\n",
        "The rational behavior at any given time depends on the following four factors:\n",
        "<ol><li>The performance measure that defines the criterion of a succesful outcome</li>\n",
        "<li>Any prior knowledge the agent has about its environment</li>\n",
        "<li>The possible actions that an agent can take</li>\n",
        "<li>The percept sequence to date that the agent has</li></ol>\n",
        "The <b>definition of a rational agent</b> is that for any given possible percept sequence, a rational agent is one that will select an action that is expected to maximize\n",
        "its performance measure using its built-in knowledge and evidence obtained from the percept sequence so far. An example of this is the vaccum cleaner:\n",
        "<ol><li>The performance measure awards one point for each clean square over a \"lifetime\" of 1000 steps.</li>\n",
        "<li>The \"geography\" or overlal dimensions of the space are known <i>a priori</i> but the dirt distribution and locations are not.</li>\n",
        "<li>The possible actions are <i>Right</i>, <i>Left</i>, and <i>Suck</i>. The former two move the agent one square in the given direction except when doing so takes the\n",
        "agent outside of its environment; it stays in place instead.</li>\n",
        "<li>The agent correctly perceives its location and if a square has dirt.</li></ol>\n",
        "One should be careful not to confuse rationality with <b>omniscience</b> or the state of knowing everything. Rationality aims to maximize the <i>expected</i> performance\n",
        "and depends only on the percept sequence <i>to date</i>, while perfection would maximize the <i>actual</i> performance. If some freak accident were to happen later on in the\n",
        "percept sequence, this is not necessarily known to a rational agent. This does not mean, however, that a rational agent should be not encouraged to engage in directly\n",
        "unintelligent activities. A rational agent is still expected to do actions leading to <b>information gathering</b> as in getting the best information possible to modify future\n",
        "percepts - like looking both ways before crossing the street. A rational agent can thus be expected to <b>explore</b> but also to <b>learn</b> as much as possible from what\n",
        "it perceives. There is an expectation that a rational agent can behave <b>autonomously</b>, that is, it is able to independently figure out how to compensate for whatever\n",
        "lack of knowledge it has initially by learning as much as possible from its percept sequence.\n",
        "\"\"\"\n",
        "chapter2_3 = \"\"\" Task environments can be characterized by several dimensions which influence how agents interact. These are:\n",
        "<ul>\n",
        "<li><b>Fully</b> vs. <b>partially</b> observable: Is the agent capable of perceiving the entire environment at any given time? An environment is fully observable if sensors are\n",
        "able to perceive all <i>relevant</i> information based on their performance measure. Oftentimes, environments are partially observable because of noise or other limitations\n",
        "that restrict the agent's sensors. If there are no sensors at all the environment is <b>unobservable</b>.</li>\n",
        "<li><b>Single-agent</b> versus <b>multiagent</b>: This relates to if there is a single agent playing a game or there are other agents involved. There are some subtle issues\n",
        "to consider: how do agents treat each other for example? In a <b>competitive</b> environment like chess, they may be trying to minimize the performance measure of other agents\n",
        "while in a <b>cooperative</b> setting like self-driving cars they may try to maximize each other's performance measure.</li>\n",
        "<li><b>Deterministic</b> and <b>nondeterministic</b>:A deterministic task environment is one where the next state is dictated completely by the current state and the action\n",
        "that the agent takes. Many partially observable environments appear to be nondeterministic, as are many real-world situations. <b>Stochastic</b> is another word that is used\n",
        "for nondeterministic, but refers to specific environments where outcomes are dictated by chance or probability.</li>\n",
        "<li><b>Episodic</b> vs. <b>sequential</b>: In an episodic task environment, each exerpeience can be segmented into distinct episodes, e.g. most classification tasks. If an \n",
        "assembly line identifies one product as defective, this has no effect on how it categorizes the next. Conversely, sequential environments are ones where the current decision\n",
        "affects all future decisions; e.g. a move taken in a game of chess affects all future moves.</li>\n",
        "<li><b>Static</b> and <b>dynamic</b>: An environment that does not change while the agent makes its choice is static, otherwise the task environment is dynamic. Static\n",
        "environments are simpler to perceive as the agent is not restricted by an amount of time to make a decision. Dynamic environments require the agent to constantly re-evaluate\n",
        "its environment. There are also <b>semidynamic</b> environments like chess where the chessboard does not change but there is a penalty associated with deciding a move.</li>\n",
        "<li><b>Discrete</b> vs. <b>continuous</b>: These relate to the <i>state</i> of the environment, the granularity at which <i>time</i> is handled and both the <i>percepts</i>\n",
        "and <i>actions</i> of agents are described. In a continuous environment, there are a range of values that change smoothly over time while discrete environments are fixed.</li>\n",
        "<li><b>Known</b> vs. <b>unknown</b>: In the former, the outcome (or probabilities) of all actions are given; otherwise, the agent will have to learn these.</li>\n",
        "</ul>\"\"\"\n",
        "chapter2_4 = \"\"\"\n",
        "There are four basic types of systems underlying all intelligent systems:\n",
        "<ol><li>A <b>simple reflex agent</b> selects actions based on its <i>current</i> precept, ignoring the rest of its history. It follows a <b>condition-action role</b> which is\n",
        "a statement of the form <b>if</b> <i>x</i> <b>then</b> <i>y</i>. This works best only in scenarios where the correct decision can be made on the basis of the current precept,\n",
        "i.e. a fully observable environment. Also can lead to infinite loops - <b>randomization</b> can escape\n",
        "loops but more sophisticated agents can handle better</li>\n",
        "<li>A model-based reflex agent can keep track of the part of the world it can no longer see by maintaining an <b>internal state</b> which holds its precept's memory. This\n",
        "requires knowledge about the world in the <b>transition model</b> and information about how the state of the world is affected by actions in the <b>sensor model</b>. An agent\n",
        "that uses such models together is a <b>model-based agent</b></li> \n",
        "<li><b>Goal based agents</b> know about the <b>goal</b> information or what state is ultimateily desirable. Through methods like <b>search</b> or <b>planning</b> they exhaust\n",
        "different possibilities, checking after to see ifthe goal-state has been reached. This is more complicated than the previous two because it considers not just the immediate\n",
        "consequence of an action but whether that action benefits the agent or not.</li>\n",
        "<li><b>Utility-based agents</b> evaluate different states to see how much <b>utility</b> they generate for an agent. They evaluate this using the <b>utility function</b>,\n",
        "an internalization of the performance measure. Since this is not known always, these maximize the <b>expected utility</b> of outcomes - the value of\n",
        "all the outcomes weighted by their probability. In some cases, a <b>model-free agent</b> can decide what action based on utility without storing a model.</li>\n",
        "</ol>\n",
        "<b>Learning agents</b> use a distinct <b>learning element</b> to improve and a <b>performance element</b> for selecting external actions. The former takes feedback\n",
        "from the <b>critic</b> and decides how to perform in the future. It uses a <b>problem generator</b> to expand possibilities.\n",
        "Representations can be structured as:\n",
        "<ul><li>in an <b>atomic representation</b> , each state of the world is indivisible - there is no internal structure, just a black box, e.g. MDP</li>\n",
        "<li>a <b>factored representation</b> breaks fixed state into <b>variables</b> or <b>attributes</b> which each have a <b>value</b></li>\n",
        "<li>to capture relationship, need a <b>structured rpresentation</b> in which objects have relationships that can be described. also of note is concept of <b>expressiveness</b>:\n",
        "a more expressive representation can capture everything a less expressive one can</li>\n",
        "<li><b>localist representation</b> implies physical mapping of concept to memory</li>\n",
        "<li>a <b>distributed representation</b> involves multiple memory locations for multiple different concepts</li></ul>\"\"\"\n",
        "\n",
        "# Lecture notes \n",
        "module1_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "print(\"The length in words is\", len(chapter2_4.split(' ')))\n",
        "print(\"The length in characters is\", len(chapter2_4))\n",
        "\n",
        "\n",
        "# Definitions of concepts from Module 1.\n",
        "# You can define any bolded term in the textbook, \n",
        "# but you most use your own words for the definitions.\n",
        "# Each definition should be at most 20 words long.\n",
        "module1_definitions = {\n",
        "\"agent\" : \"anything that can perceive and act in an environment\",\n",
        "\"actuators\" : \"devices that allow agent to act on information from sensors\",\n",
        "\"environment\" : \"the surroundings of an agent and everything it interacts with\",\n",
        "\"sensor\" : \"a device that can pick up a precept\",\n",
        "\"percept\" : \"the content being perceived by an agent\",\n",
        "\"percept sequence\" : \"the complete history of all the percepts perceived by an agent\",\n",
        "\"agent function\" : \"a function that maps any percept sequence to a respective action for the agent\",\n",
        "\"agent program\" : \"a program that implements the agent function\",\n",
        "\"rational agent\" : \"an agent that takes the ractional action\",\n",
        "\"consequentialism\" : \"a notion in philosophy that judges agents by the consequences of their actions\",\n",
        "\"performance measure\" : \"a function that evaluates a sequence of states in the environment\",\n",
        "\"explore\" : \"the process of gathering percepts in an unknown environment\",\n",
        "\"information gathering\" : \"the decision to do actions in order to modify future percepts\",\n",
        "\"learn\" : \"to gain new knowledge about an environment based on percepts\",\n",
        "\"autonomy\" : \"the ability to learn from percepts to compensate for incomplete or partial initial knowledge\"\n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 401\n",
            "The length in characters is 2992\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "elCCgY64spuq"
      },
      "source": [
        "## Preview of your notes for Module 1\n",
        "\n",
        "Here's a preview of your how your notes will be displayed.  At the end of this Python notebook, we'll aggregate your notes for all modules together into a single display.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gYYeNcdM_maE",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 3243
        },
        "cellView": "form",
        "outputId": "8e060547-c5f5-447d-f17e-393511f2a031"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "from IPython.display import display, Markdown, Latex\n",
        "\n",
        "module1_sections = [\n",
        "  (\"AIMA Chapter 1.1 What is AI?\", chapter1_1),\n",
        "  (\"AIMA Chapter 27.1 The Limits of AI\", chapter27_1),\n",
        "  (\"AIMA Chapter 27.1 Can Machines Really Think?\", chapter27_2),\n",
        "  (\"AIMA 2.1 Agents and Environments\", chapter2_1),\n",
        "  (\"AIMA 2.2 Good Behavior: The Concept of Rationality\", chapter2_2),\n",
        "  (\"AIMA 2.3 The Nature of Environments\", chapter2_3),\n",
        "  (\"AIMA 2.4 The Structure of Agents\", chapter2_4),\n",
        "  (\"Module 1 Lecture Notes\", module1_lecture_notes),\n",
        "]\n",
        "\n",
        "\n",
        "def length_check(sections):\n",
        "  warnings = \"\"\n",
        "  for section, summary in sections:\n",
        "    max_word_length = 500\n",
        "    max_character_length = 3000\n",
        "    if \"lecture\" in section.lower():\n",
        "      # It's OK to do longer summaries of the lectures\n",
        "      max_word_length = 2000\n",
        "      max_character_length = 12000  \n",
        "    if (len(summary) > max_character_length or len(summary.split(' ')) > max_word_length):\n",
        "      warnings += \"WARNING: Your summary of '{section}' is too long!\\n\".format(section=section)\n",
        "  return warnings\n",
        "\n",
        "def length_check_definitions(definitions):\n",
        "  warnings = \"\"\n",
        "  for concept, definition in definitions.items():\n",
        "    if(len(definition.split(' ')) > 20):\n",
        "      warnings += \"WARNING: Your definition of '{concept}' is too long!\\n\".format(concept=concept)\n",
        "  return warnings\n",
        "\n",
        "\n",
        "def format_module(module_num, title, sections, definitions, show_placeholder=False):\n",
        "  markdown = \"\"\n",
        "\n",
        "  # Check summary lengths\n",
        "  warnings = length_check(sections)\n",
        "  if warnings != \"\":\n",
        "    markdown += \"## \" + warnings\n",
        "\n",
        "  # Check definition lengths\n",
        "  warnings = length_check_definitions(definitions)\n",
        "  if warnings != \"\":\n",
        "    markdown += \"## \" + warnings\n",
        "\n",
        "  markdown += \"# Module {num}: {title}\\n\".format(num=module_num, title=title)\n",
        "\n",
        "\n",
        "  for section, summary in sections:\n",
        "    if show_placeholder and summary == \" Your summary here \":\n",
        "      summary = placeholder\n",
        "    markdown += \"## {section}\\n{summary}\\n\\n\".format(section=section, summary=summary)\n",
        "  \n",
        "  markdown += \"## Module {num} Definitions\\n\".format(num=module_num)\n",
        "  for concept, definition in definitions.items():\n",
        "    markdown += \"* **{concept}** - {definition}\\n\".format(concept=concept.capitalize(), definition=definition)\n",
        "  return markdown\n",
        "\n",
        "\n",
        "module1_markdown = format_module(1, \"Rational Agents and Task Environments\", module1_sections, module1_definitions)\n",
        "display(Markdown(module1_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 1: Rational Agents and Task Environments\n## AIMA Chapter 1.1 What is AI?\n There is some debate over the meaning of being intelligent, leading to the following four approaches:\n<table>\n<tr><td></td><td><b>Thinking</b></td><td><b>Acting</b></td></tr>\n<tr>\n<td><b>Humanly</b></td>\n<td>We can model how humans think in three main ways:\n<ul><li><b>introspection</b> - monitoring our own thoughts</li>\n<li><b>psychological experiments</b> - observing people in action</li>\n<li><b>brain imaging</b> - observing the brain in action</li></ul>\nThese developments have led to fields like <b>cognitive science</b> which use experimental techniques from psychology, and some of these brain models have made their way into\ncomputer vision research.</td>\n<td>The <b>Turing test</b> asks if a machine can appear human to a human interrogator. In order to do this, a Turing machine would need the following abilities:\n<ul><li><b>natural language processing</b> to be able to communicate the way a human does</li>\n<li><b>knowledge representation</b> to store and reason about what it perceives</li>\n<li><b>automated reasoning</b> to respond to questions and come up with new conclusions</li>\n<li><b>machine learning</b> to extrapolate patterns to apply to new situations</ul>\nTo apss the more complicated <b>total Turing test</b> which requires interacting with people, it would also need:\n<ul><li><b>computer vision</b> to perceive objects in the real world</li>\n<li><b>robotics</b> to move and be able to manipulate objects</li></ul></td></tr>\n<tr>\n<td><b>Rationally</b></td>\n<td>The \"Laws of Thought\" approach is modeled after attempts to codify \"right thinking\" by philosophers like Aristotle. He defined certain <b>syllogisms</b> for\nlogical reasoning that formed the basis for the field of <b>logic</b>. In efforts since these are codified into a formal language via the <b>logicist</b> tradition.\nWhen outcomes cannot be defined with <i>certainty</i>, the theory of <b>probability</b> allows rigorous reasoning with information that is uncertain.</td>\n<td>An <b>agent</b> is something that acts. Computer argents are expected to operate autonomously by perceiving its\nenvironment, persisting over an extended time period, adapting to change, and identifying and pursuing goals. A <b>rational agent</b> is one\nthat acts in a way that achieves the best expected outcome when given an uncertain situation.This has the advantage that it is more general than the\n\"laws of thought\" approach as there are several ways to achieve rationality beyond inference and it is amenable to the scientific process as it can\nbe rigorously defined and generalized. Specific agent designs and goals are less ambiguous to define than mimicking human behavior.</td></tr>\n</table><br />\nAs AI becomes more complicated, the specific goals and objectives are more ambiguous to define. How should a self-driving car\nbehave to avoid endangering other errant drivers, passenger comfort, etc? These lead to the <b>value alignment problem</b> where we ask if the values\nin the AI match those of humans.\n\n## AIMA Chapter 27.1 The Limits of AI\nJohn Searle, a philosopher, proposed the concept of <b>strong AI</b> and <b>weak AI</b>, where the former denotes machines that are actually\ncapable of human thought and the altter referring to machines that merely act like humans. The notion of weak AI has since been disputed and indeed, Alan Turing anticipated\nseveral of the main arguments that have been made against AI:\n<ol>\n<li>Turing's \"argument from the informality of behavior\" stated that humans do not have a formal set of rules and instead use informal guidance to make decisions. This view\nwas further argued for by Dreyfus who wrote books on what AI couldn't do, but these models have since been discarded as the idea of <b>Good Old-Fashioned AI (GOFAI)</b>.\nIndeed, while the <b>qualification problem</b> has shown that some problems are difficult to explicitly model, deep learning and other techniques have proven to be able\nto structure informal situations. Another variant of this argument is <b>embodied cognition</b>: cognition takes place throughout the whole body, placed in an environment.\nThen, we only need to study the agent's behavior as a whole - under such a view, robotics, vision, etc. become central as opposed to peripheral.</li>\n<li>The \"argument from disability\" states that a machine can never do certain things - Turing listed \"tell right from wrong, make mistakes, fall in love, enjoy strawberries\nand cream, make someone fall in love with it, learn from experience, etc.\" Computers have shown to be able to do many of these things, including making significant\ncontributions to fields like chemistry, computer science, even creating new art via style transfer.</li>\n<li>Gobel stated that for any axiomatic framework <i>F</i> that is capable of doing arithmetic, one can construct\na Godel sentence <i>G(F)</i> which ahd two properties:\n<ul><li><i>G(F)</i> is a sentence of <i>F</i> that cannot be proved within <i>F</i>\n<li>if <i>F</i> is consistent, then it follows that <i>G(F)</i> is true.</li></ul>\nIt follows from this that by the incompleteness theorem, machines are limited and cannot establish their own Godel sentence. However, this has been contended with for a couple\nof reasons:\n<ol><li>Just because one agent, e.g. human cannot assert something, that does not mean other people or machines can</li>\n<li>Godel's incompleteness theorem apply to <i>mathematics</i> and not necessarily <i>computers</i> as no one is capable of proving something that is impossible to prove.\nHumans sometimes assume their own consistency only to be proven false later, as in the four-color map problem.</li>\n<li>This fails to consider finite limitations for computers and human brains, who can be described as very large propositional logic systems. But just as a human can change\nits decision, a computer may have different results with machine learning or software rewriting.</li></ol>\n</ol>\nThe <b>turing test</b> has not been of much focus in the AI community although some Chatbots have shown a lot of potential.\n\n\n## AIMA Chapter 27.1 Can Machines Really Think?\nSome further debate has gone into whether it is even important that machines think - for example, submarines don't necessarily swim and airplanes don't\nnecessarily fly. Turing himself suggested the notion of <b>polite convention</b> - that is, we take what other people say at face value and don't try to break apart their\ninner thoughts or how they reason about what they say, so similar leniencey should be given to machines. There have been some philosophers who disagree with this - Searle\nalso came up with a thoguht experiment called the <b>Chinese room</b>, in which a translator looks up every single English word that comes in and presses a button resulting\nin a correct Chinese translation on the other end. Searle argues that the translator does not in fact know Chinese even though the end user would be getting perfect\nChinese translations. Searle was thus a big proponent of <b>biological naturalism</b> which states that mental states are caused by low-level biological processes in the\nneurons themselves and the intrinsic properties of these are what define intelligence. Thus much of this debate surrounds the concept of <b>consciousness</b> or awareness\nof the outside world, and the subjective experience of living, e.g. the concept of self. The <b>qualia</b> or intrinsic nature of these experiences is not so important\naccording to Turing, and further some of these aspects can be programmed into machines but this <i>awareness</i> is still mostly a part of the human experience.\n\n## AIMA 2.1 Agents and Environments\nHere, an <b>agent</b> is any object that is capable of perceiving its <b>environment</b> with the use of <b>sensors</b> and then acting on these <b>percepts</b>\nusing <b>actuators</b>. Some examples of agents include a piece of software which receives packets and I/O from a user and then sends packets or displays information in\nresponse, or a robot that perceives through its camera lens and infrared range finders and then moves itself based on the environment it perceives. In general, an agent\nconsiders its entire <b>percept sequence</b> and the agent's choice of what action to take at any given instant depends on any built-in knowledge provided and on the\npercept sequence up to the current moment, and not on anything it has yet to perceive. The way that an agent behaves can be described by its <b>agent function</b> which maps\na percept sequence to a specific action.\n<br /><br />\nOne interpretation of the agent function is that it <i>tabulates</i>the agent function - but such a table could be infinitely large and this is an <i>external</i> representation\nof the agent's choices. <i>Internally</i>, the agent function is implemented by the <b>agent program</b>. The key difference between the two is that the agent function is an\nabstract mathematical function while the agent program is its actual implementation.\n<br /><br />\nA final note is that the idea of an agent is meant to be a paradigm or tool for analyzing a system as opposed to an absolute characterization of the system. There is no\ninherent dichotomy between agents and non-agents in the real world. In some sense, most of engineering involves artifacts that interact the world and artifical intelligence\noperates at the end of the spectrum where resources have significant computational resources and are tasked with making nontrivial decisions.\n\n## AIMA 2.2 Good Behavior: The Concept of Rationality\nA <b>rational agent</b> can be defined as one that chooses the right action. But in terms of what defines \"right\", AI tends to stick to the notion of\n<b>consequentialism</b>, or evaluating its success by the consequences of its actions. The <b>performance measure</b> is a function that evaluates any given sequence of successive\nenvironment states. Since machines do not have desires or preferences, at least not initially, it is better in general to design performance measures based on the outcome one wants to achieve in an environment (the goal state) as opposed to how one thinks \nthat an agent <i>should</i> behave.\n<br /><br />\nThe rational behavior at any given time depends on the following four factors:\n<ol><li>The performance measure that defines the criterion of a succesful outcome</li>\n<li>Any prior knowledge the agent has about its environment</li>\n<li>The possible actions that an agent can take</li>\n<li>The percept sequence to date that the agent has</li></ol>\nThe <b>definition of a rational agent</b> is that for any given possible percept sequence, a rational agent is one that will select an action that is expected to maximize\nits performance measure using its built-in knowledge and evidence obtained from the percept sequence so far. An example of this is the vaccum cleaner:\n<ol><li>The performance measure awards one point for each clean square over a \"lifetime\" of 1000 steps.</li>\n<li>The \"geography\" or overlal dimensions of the space are known <i>a priori</i> but the dirt distribution and locations are not.</li>\n<li>The possible actions are <i>Right</i>, <i>Left</i>, and <i>Suck</i>. The former two move the agent one square in the given direction except when doing so takes the\nagent outside of its environment; it stays in place instead.</li>\n<li>The agent correctly perceives its location and if a square has dirt.</li></ol>\nOne should be careful not to confuse rationality with <b>omniscience</b> or the state of knowing everything. Rationality aims to maximize the <i>expected</i> performance\nand depends only on the percept sequence <i>to date</i>, while perfection would maximize the <i>actual</i> performance. If some freak accident were to happen later on in the\npercept sequence, this is not necessarily known to a rational agent. This does not mean, however, that a rational agent should be not encouraged to engage in directly\nunintelligent activities. A rational agent is still expected to do actions leading to <b>information gathering</b> as in getting the best information possible to modify future\npercepts - like looking both ways before crossing the street. A rational agent can thus be expected to <b>explore</b> but also to <b>learn</b> as much as possible from what\nit perceives. There is an expectation that a rational agent can behave <b>autonomously</b>, that is, it is able to independently figure out how to compensate for whatever\nlack of knowledge it has initially by learning as much as possible from its percept sequence.\n\n\n## AIMA 2.3 The Nature of Environments\n Task environments can be characterized by several dimensions which influence how agents interact. These are:\n<ul>\n<li><b>Fully</b> vs. <b>partially</b> observable: Is the agent capable of perceiving the entire environment at any given time? An environment is fully observable if sensors are\nable to perceive all <i>relevant</i> information based on their performance measure. Oftentimes, environments are partially observable because of noise or other limitations\nthat restrict the agent's sensors. If there are no sensors at all the environment is <b>unobservable</b>.</li>\n<li><b>Single-agent</b> versus <b>multiagent</b>: This relates to if there is a single agent playing a game or there are other agents involved. There are some subtle issues\nto consider: how do agents treat each other for example? In a <b>competitive</b> environment like chess, they may be trying to minimize the performance measure of other agents\nwhile in a <b>cooperative</b> setting like self-driving cars they may try to maximize each other's performance measure.</li>\n<li><b>Deterministic</b> and <b>nondeterministic</b>:A deterministic task environment is one where the next state is dictated completely by the current state and the action\nthat the agent takes. Many partially observable environments appear to be nondeterministic, as are many real-world situations. <b>Stochastic</b> is another word that is used\nfor nondeterministic, but refers to specific environments where outcomes are dictated by chance or probability.</li>\n<li><b>Episodic</b> vs. <b>sequential</b>: In an episodic task environment, each exerpeience can be segmented into distinct episodes, e.g. most classification tasks. If an \nassembly line identifies one product as defective, this has no effect on how it categorizes the next. Conversely, sequential environments are ones where the current decision\naffects all future decisions; e.g. a move taken in a game of chess affects all future moves.</li>\n<li><b>Static</b> and <b>dynamic</b>: An environment that does not change while the agent makes its choice is static, otherwise the task environment is dynamic. Static\nenvironments are simpler to perceive as the agent is not restricted by an amount of time to make a decision. Dynamic environments require the agent to constantly re-evaluate\nits environment. There are also <b>semidynamic</b> environments like chess where the chessboard does not change but there is a penalty associated with deciding a move.</li>\n<li><b>Discrete</b> vs. <b>continuous</b>: These relate to the <i>state</i> of the environment, the granularity at which <i>time</i> is handled and both the <i>percepts</i>\nand <i>actions</i> of agents are described. In a continuous environment, there are a range of values that change smoothly over time while discrete environments are fixed.</li>\n<li><b>Known</b> vs. <b>unknown</b>: In the former, the outcome (or probabilities) of all actions are given; otherwise, the agent will have to learn these.</li>\n</ul>\n\n## AIMA 2.4 The Structure of Agents\n\nThere are four basic types of systems underlying all intelligent systems:\n<ol><li>A <b>simple reflex agent</b> selects actions based on its <i>current</i> precept, ignoring the rest of its history. It follows a <b>condition-action role</b> which is\na statement of the form <b>if</b> <i>x</i> <b>then</b> <i>y</i>. This works best only in scenarios where the correct decision can be made on the basis of the current precept,\ni.e. a fully observable environment. Also can lead to infinite loops - <b>randomization</b> can escape\nloops but more sophisticated agents can handle better</li>\n<li>A model-based reflex agent can keep track of the part of the world it can no longer see by maintaining an <b>internal state</b> which holds its precept's memory. This\nrequires knowledge about the world in the <b>transition model</b> and information about how the state of the world is affected by actions in the <b>sensor model</b>. An agent\nthat uses such models together is a <b>model-based agent</b></li> \n<li><b>Goal based agents</b> know about the <b>goal</b> information or what state is ultimateily desirable. Through methods like <b>search</b> or <b>planning</b> they exhaust\ndifferent possibilities, checking after to see ifthe goal-state has been reached. This is more complicated than the previous two because it considers not just the immediate\nconsequence of an action but whether that action benefits the agent or not.</li>\n<li><b>Utility-based agents</b> evaluate different states to see how much <b>utility</b> they generate for an agent. They evaluate this using the <b>utility function</b>,\nan internalization of the performance measure. Since this is not known always, these maximize the <b>expected utility</b> of outcomes - the value of\nall the outcomes weighted by their probability. In some cases, a <b>model-free agent</b> can decide what action based on utility without storing a model.</li>\n</ol>\n<b>Learning agents</b> use a distinct <b>learning element</b> to improve and a <b>performance element</b> for selecting external actions. The former takes feedback\nfrom the <b>critic</b> and decides how to perform in the future. It uses a <b>problem generator</b> to expand possibilities.\nRepresentations can be structured as:\n<ul><li>in an <b>atomic representation</b> , each state of the world is indivisible - there is no internal structure, just a black box, e.g. MDP</li>\n<li>a <b>factored representation</b> breaks fixed state into <b>variables</b> or <b>attributes</b> which each have a <b>value</b></li>\n<li>to capture relationship, need a <b>structured rpresentation</b> in which objects have relationships that can be described. also of note is concept of <b>expressiveness</b>:\na more expressive representation can capture everything a less expressive one can</li>\n<li><b>localist representation</b> implies physical mapping of concept to memory</li>\n<li>a <b>distributed representation</b> involves multiple memory locations for multiple different concepts</li></ul>\n\n## Module 1 Lecture Notes\n Your summary here \n\n## Module 1 Definitions\n* **Agent** - anything that can perceive and act in an environment\n* **Actuators** - devices that allow agent to act on information from sensors\n* **Environment** - the surroundings of an agent and everything it interacts with\n* **Sensor** - a device that can pick up a precept\n* **Percept** - the content being perceived by an agent\n* **Percept sequence** - the complete history of all the percepts perceived by an agent\n* **Agent function** - a function that maps any percept sequence to a respective action for the agent\n* **Agent program** - a program that implements the agent function\n* **Rational agent** - an agent that takes the ractional action\n* **Consequentialism** - a notion in philosophy that judges agents by the consequences of their actions\n* **Performance measure** - a function that evaluates a sequence of states in the environment\n* **Explore** - the process of gathering percepts in an unknown environment\n* **Information gathering** - the decision to do actions in order to modify future percepts\n* **Learn** - to gain new knowledge about an environment based on percepts\n* **Autonomy** - the ability to learn from percepts to compensate for incomplete or partial initial knowledge\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9xRbdqDox2zD"
      },
      "source": [
        "#Module 2 - Python"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DpBvu7EMx7sy"
      },
      "source": [
        "# No readings for this module.  Instead you can summarize the lecture instead.\n",
        "\n",
        "# Lecture notes \n",
        "module2_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Each definition should be at most 20 words long.\n",
        "module2_definitions = {\n",
        "\"mutability\" : \"your definition here\",\n",
        "}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ydOK__lwWgdp",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 207
        },
        "outputId": "7e93cc4c-d828-43b1-cc10-2ecb26671896"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "\n",
        "module2_sections = [\n",
        "  (\"Module 2 Lecture Notes\", module2_lecture_notes),\n",
        "]\n",
        "\n",
        "module2_markdown = format_module(2, \"Python\", module2_sections, module2_definitions)\n",
        "display(Markdown(module2_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 2: Python\n## Module 2 Lecture Notes\n Your summary here \n\n## Module 2 Definitions\n* **Mutability** - your definition here\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "46MQLR-Gx8dZ"
      },
      "source": [
        "#Module 3 - Search Problems"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BN0b1eVYx-nd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9b124c53-32a7-482c-9f7c-33aa94a413bb"
      },
      "source": [
        "# Russell and Norvig, AIMA 4th Edition, AIMA Chapter 3 “Problem Solving by Search” (3.1-3.4)\n",
        "chapter3_1 = \"\"\"An agent that is capable of planning ahead and considering sequences of actions to find a potential solution that reaches a certain desired goal state can be\n",
        "described as a <b>problem-solving agent</b>. It undertakes the process of <b>search</b> that can be either <b>informed</b> or <b>uninformed</b> depending on if the estimate\n",
        "to the goal state is known. If the situation is completely <b>unknown</b> then an agent can do no better at random guessing, but as information becomes available, the agent\n",
        "is capable of the following:\n",
        "<ul><li><b>Goal formation</b>: The agent defines what its primary <b>goal</b> is. This is important as it dictates and organizes the agent's future behavior and restricts\n",
        "which potential moves may be considered.</li>\n",
        "<li><b>Problem formulation</b>: The agent comes up with a description of the different states in the environment and the actions that are needed to make the goal, i.e.\n",
        "an abstract representation of the world.</li>\n",
        "<li><b>Search</b>: The agent will simulate multiple different paths to see if it can find a <b>solution</b>, or a sequence of actions that lead to goal state, if one exists.\n",
        "</li><li><b>Execution</b>: The agent evaluates the actions from its solution, one at a time.</li></ul>\n",
        "The environment can be further characterized as <b>open-loop</b> or <b>closed-loop</b> if the environment is nondeterministic.\n",
        "<br /><br />\n",
        "A search <b>problem</b> is formally defined as having the following:\n",
        "<ul><li><b>State space</b>: the set of all possible <b>states</b> of the environment.</li>\n",
        "<li><b>Initial state</b>: the initial state that the agent is initially in</li>\n",
        "<li><b>Goal state(s)</b>: the target state that the agent attempts to reach, can be multiple. A method e.g. <i>Is-Goal(s)</i> should return boolean indicating if reached.</li>\n",
        "<li><b>Action(<i>s</i>)</b>: a method which returns a finite set of actions that can be executed given a state <i>s</i></li>\n",
        "<li><b>Transition model</b>: a method which denotes the result of each action, e.g. <i>Results(s,a)</i> returns the result of applying action <i>a</i> to state <i>s</i>.</li>\n",
        "<li><b>Action cost function</b> <i>c(s,a,s')</i> which denotes the numeric cost of applying an action <i>a</i> on state <i>s</i> to achieve state <i>s'</i>.\n",
        "This cost function should reflect the agent's own performance measure.</li></ul>\n",
        "Any sequence of actions leads forms a <b>path</b>, a path that leads to the goal state is a <b>solution</b> and an <b>optimal solution</b> is one that minimizes the path cost.\n",
        "We can also visualize the solution in a <b>graph</b> in which edges are actions between the vertices which are states.\n",
        "<br /><br />\n",
        "One final thing to consider is that these representations are a <b>model</b> and these have a specific <b>level of abstraction</b> that removes as much information as possible\n",
        "while ensuring that <i>validity</i> is preserved and that abstract solutions can be carried out for the agent. The process of removing these other details is <b>abstraction</b>.\"\"\"\n",
        "chapter3_2 = \"\"\" It is important first to distinguish between a <b>standardized problem</b> and actual <b>real-world problems</b>. Some examples of this include:\n",
        "<ul><li><b>State space</b>: the set of all possible cells. For <b>vacuum world</b>, if there are <i>n</i> cells there are <i>n * 2<sup>n</sup></i> states. This can also be a\n",
        "(partially filled) Sudoku board, a description of all the tiles in the <b>sliding-tile puzzle</b>, a <b>sobokan board</b>, the real numbers in Knuth's conjecture, or the\n",
        "location (e.g. an airport) and current time in the <b>route-finding problem</b>. The Sobokan example can have <i>n * n!/(b!(n-b)!)</i> or over 200 trillion possibilities\n",
        "for an 8x8 board, Knuth's conjecture has infinite possible states.</li>\n",
        "<li><b>Initial state</b>: Any state in <b>grid world</b> or any arrangement of <b>sliding-tiles</b> (note that a goal can be reached by about half of the states),\n",
        "the number 4 in Knuth's conjecture, or the user's home airport.</li>\n",
        "<li><b>Goal state(s)</b>: For <b>vacuum world</b>, the state in which every cell is clean. For <b>sliding-puzzle</b>, any state, but typically that in which the numbers are in\n",
        "order, for Knuth's conjecture, the desired positive integer, and for <b>route planning</b> a specified destination city.</li>\n",
        "<li><b>Action(<i>s</i>)</b>: These can be <b>absolute</b> movement actions like <i>Upward, Downward</i> or <b>egocentric actions</b> with respect to the agent, like\n",
        "<i>Forward</i>, <i>Backward</i>, etc. for the vacuum world. For the <b>sliding-tile puzzle</b>, this involves moving a tile in one of the cardinal directions to an empty\n",
        "space. For Knuth's conjecture, this can be either the square root, floor, or factorial operations. And for the <b>route-finding</b> problem this is taking any flight\n",
        "from the current location that leaves after the current time and leaves enough time for a within-airport transfer.</li>\n",
        "<li><b>Transition model</b>: certain actions like <i>Suck</i> remove dirt from one cell, resulting in a new state, and motion activities leave the vacuum in a different\n",
        "position. The puzzle pieces move into the empty space if possible, creating a new empty space. The transition model for Knuth's conjecture is the mathematicaloperations,\n",
        "and for the route planning problem, it will be the new destination city and flight's arrival time.</li>\n",
        "<li><b>Action cost function</b> Each action costs 1 for the <b>vacuum world</b>, <b>sliding-tile puzzle</b> and Knuth's conjecture problems. For the <b>route planning</b>\n",
        "algorithm, this can be a combination of cost (monetary), time (either waiting and/or flying), and other factors like seat quality, flight points, etc.</li></ul>\n",
        "Some other more complicated problems that can be solved include the <b>traveling salesperson problem (TSP)</b> which is a type of <b>touring problem</b> in which every city on\n",
        "a map is to be visited once and <b>VLSI layout</b> which groups cells into an ideal <b>cell layout</b> and then designing a <b>channel routing</b> procedure.\n",
        "\"\"\"\n",
        "chapter3_3 = \"\"\"A <b>search algorithm</b> takes as input a search problem and returns either a solution or failure. A <b>search tree</b> is a representation fo the paths\n",
        "taken from the initial state over the state-space graph that reach towards a goal state. Each <b>node</b> in the search tree corresponds to a state in the search tree and\n",
        "its edges correspond to actions from that node. From the root node, we <b>expand</b> each node by considering the available ACTION and RESULT functions to see what other\n",
        "<b>child node</b> or <b>successor node</b> can be generated from the current node - now the <b>parent node</b>. The nodes in the search tree which have been reached already\n",
        "are the <b>frontier</b> and as a node gets added to this it is considered <b>reached</b>. The frontier hence <b>separates</b> two regions of the state-space graph: an\n",
        "interior where every node has been expanded, and the exterior which has not been reached.\n",
        "<br /><br />\n",
        "One such search algorithm is <b>best-first search</b> in which the node <i>n</i> that is chosen has the minimum value of an <b>evaluation fucntion</b> <i>f(n)</i>. The\n",
        "algorithm proceeds on each iteration by checking if it is the goal state (and returning if so), or expanding on its neighbors otherwise. Each node is added to the frontier\n",
        "if it has not been reached before or if it has a lower path cost than previously. The algorithm returns either a path to the goal or indicates failure.\n",
        "<br /><br />\n",
        "A <b>node</b> in a tree is represented by a data structure which has four components:\n",
        "<ul><li><i>node</i>.STATE: the state the node corresponds with</li>\n",
        "<li><i>node</i>.PARENT: the node from which the node was generated</li>\n",
        "<li><i>node</i>.ACTION: the action from which the node was reached (from the parent node)</li>\n",
        "<li><i>node</i>.PATH-COST: the total cost along path from initial state to the node, synonymous to <i>g(node)</i></li></ul>\n",
        "Following the PARENT pointers back allow one to recover the entire states and actions leading to a node. Doing this from the goal leads to the solution.\n",
        "<br /><br />\n",
        "The <b>frontier</b> needs to support operations to check if it IS-EMPTY, and to POP and ADD elements. Some types of <b>queues</b> that can be used include:\n",
        "<ul><li>A <b>priority queue</b> pops the element with minimum cost according to a function <i>f</i></li>\n",
        "<li>A <b>FIFO queue</b> pops the node that was added first</li>\n",
        "<li>A <b>LIFO queue</b> (or a <b>stack</b>) pops the node that was added most recently</li></ul>\n",
        "Evaluate performance on:\n",
        "<ul><li><b>Completeness</b>: guaranteed to find solution when there is one, and correctly report failure if not?</li>\n",
        "<li><b>Cost optimality</b>: guaranteed to return lowest cost solution?</li>\n",
        "<li><b>Time complexity</b>: how long does it take? measured by # of states/actions considered.</li>\n",
        "<li><b>Space complexity</b>: how much memory needed to search?</li></ul>\n",
        "Search is <b>graph search</b> if it checks for <b>redundant path</b>, otherwise <b>tree-like search</b>.\n",
        "\"\"\"\n",
        "chapter3_4 = \"\"\"Types of uninformed search include:\n",
        "<ol><li><b>Breadth-First Search</b>: a strategy in which a root node is expanded, then its successors, and their successors. Complete even on an infinite state space. \n",
        "One optimization is to do an <b>early goal test</b>, i.e. checking if a goal state has been reached as soon as it has been <i>generated</i> \n",
        "as opposed to when dequeued as in with a <b>late goal test</b>.</li>\n",
        "<li>Dijkstra's algorithm or <b>uniform cost search</b> spreads out in waves of uniform path-cost. It can be implemented as a call to BEST-FIRST-SEARCH with PATH-COST\n",
        "as the evaluation function. BFS tests for the goal state only when it expands a node, not when it generates it.</li>\n",
        "<li><b>Depth first search</b> is tree-like search, expands node until it cannot anymore, then reverts to state where there is more to explore. In\n",
        "<b>backtracking search</b>, even less memory used as it generates one successor at a time, modifying state description instead of allocating memroy for a \n",
        "brand-new state.</li>\n",
        "<li><b>Depth-limited search</b> is a version of DFS where we treat anything beyond a depth-limit <i>l</i> and treat nodes at that depth at terminal. A good heuristic\n",
        "is the <b>diameter</b> of state-space graph, but this is not always known. <b>Iterative deepening search</b> works by doing depth-limited search for 1, 2, etc. until\n",
        "a cutoff value is reached.</li>\n",
        "<li><b>Bidirectional search</b> expands from the initial state and backwards from the goal state(s). Need to\n",
        "reverse the problem and order of visited solutions for backward direction.</i>\n",
        "</ol>\n",
        "\n",
        "<table>\n",
        "<tr><td>Criterion</td><td>Breadth-First</td><td>Uniform-Cost</td><td>Depth-First</td><td>Depth-Limited</td><td>Iterative Deepening</td><td>Bidirectional</td></tr>\n",
        "<tr><td>Complete?</td><td>Yes<sup>1</sup></td><td>Yes<sup>1,2</sup></td><td>No</td><td>No</td><td>Yes<sup>1</sup></td><td>Yes<sup>1,4</sup></td></tr>\n",
        "<tr><td>Optimal cost?</td><td>Yes<sup>3</sup></td><td>Yes</td><td>No</td><td>No</td><td>Yes<sup>3</sup></td><td>Yes<sup>3,4</sup></td></tr>\n",
        "<tr><td>Time</td><td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>1+floor(C*/e)</sup>)</i></td><td><i>O(b<sup>m</sup>)</i></td><td><i>O(b<sup>l</sup>)</i></td>\n",
        "<td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>d/2</sup>)</i></td></tr>\n",
        "<tr><td>Space</td><td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>1+floor(C*/e)</sup>)</i></td><td><i>O(bm)</i></td></td><td><i>O(bl)</i></td></td><td><i>O(bd)</i></td>\n",
        "<td><i>O(b<sup>d/2</sup>)</i></td></tr>\n",
        "</table>\n",
        "where <i>b</i> is the branching factor, <i>m</i> is the maximum depth of the search tree, <i>d</i> is the depth of the shallowest solution or <i>m</i> when no solution,\n",
        "and <i>l</i> is the depth-limit.<br />\n",
        "<sup>1</sup>: complete if <i>b</i> is finite and state space is either finite or has a solution<br />\n",
        "<sup>2</sup>: complete if all actions have cost >= <i>e</i> > 0<br />\n",
        "<sup>3</sup>: cost-optimal if actions are all identical<br />\n",
        "<sup>4</sup>: if both directions are breadth-first or uniform-cost\"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module3_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "\n",
        "\n",
        "print(\"The length in words is\", len(chapter3_4.split(' ')))\n",
        "print(\"The length in characters is\", len(chapter3_4))\n",
        "\n",
        "# Definitions of concepts from Module 3.\n",
        "# You can define any bolded term in the textbook, \n",
        "# but you most use your own words for the definitions.\n",
        "# Each definition should be at most 20 words long.\n",
        "module3_definitions = {\n",
        "\"problem-solving agent\" : \"an agent that can formulate a plan via search\",\n",
        "\"atomic\" : \"representations of the world that are considered as a whole with no visible internal structure\",\n",
        "\"informed algorithm\" : \"an algorithm in which an agent has information to estimate its distance to a goal state\",\n",
        "\"uninformed algorithm\" :  \"an algorithm in which an agent has no information to estimate its distance to a goal state\",\n",
        "\"solution\" : \"a sequence of actions through which an agent can reach its goal state\",\n",
        "\"open-loop system\" : \"a system in which an agent can ignore the percepts once its solution is determined\",\n",
        "\"closed-loop system\" : \"a system in which the environment can change, requiring the agent to rely on percepts\",\n",
        "\"state space\" : \"the set of possible states that the environment can be\",\n",
        "\"initial state\" : \"the initial state of the environment that the agent starts the problem in\",\n",
        "\"goal states\" : \"states that the agent intends to achieve\",\n",
        "\"actions\" : \"the choices an agent can make\",\n",
        "\"transition model\" : \"a formula that describes what an action does\",\n",
        "\"action cost function\" : \"a function that describes the cost an agent occurs in an action\",\n",
        "\"path\" : \"a sequence of actions\",\n",
        "\"optimal solution\" : \"the solution that has the lowest path cost to the agent\",\n",
        "\"model\" : \"an abstract mathematical representation\",\n",
        "\"abstraction\" : \"the process of removing details from a representation\",\n",
        "\"level of abstraction\" : \"the degree to which detail has been omitted for an action\",\n",
        "\"standardized problem\" : \"an illustrative problem to show an example of problem-solving\",\n",
        "\"real-world problem\" : \"a problem that arises from actual use case\"\n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 306\n",
            "The length in characters is 2995\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lvtN3p7ET3Tx",
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 2477
        },
        "outputId": "bcb5a24c-7ce2-4153-873b-984855c5e279"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module3_sections = [\n",
        "  (\"AIMA Chapter 3.1 Problem-Solving Agents\", chapter3_1),\n",
        "  (\"AIMA Chapter 3.2 Example Problems\", chapter3_2),\n",
        "  (\"AIMA Chapter 3.3 Search Algorithms\", chapter3_3),\n",
        "  (\"AIMA Chapter 3.4 Uninformed Search Strategies\", chapter3_4),\n",
        "  (\"Module 3 Lecture Notes\", module3_lecture_notes),\n",
        "]\n",
        "\n",
        "module3_markdown = format_module(3, \"Search Problems\", module3_sections, module3_definitions)\n",
        "display(Markdown(module3_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 3: Search Problems\n## AIMA Chapter 3.1 Problem-Solving Agents\nAn agent that is capable of planning ahead and considering sequences of actions to find a potential solution that reaches a certain desired goal state can be\ndescribed as a <b>problem-solving agent</b>. It undertakes the process of <b>search</b> that can be either <b>informed</b> or <b>uninformed</b> depending on if the estimate\nto the goal state is known. If the situation is completely <b>unknown</b> then an agent can do no better at random guessing, but as information becomes available, the agent\nis capable of the following:\n<ul><li><b>Goal formation</b>: The agent defines what its primary <b>goal</b> is. This is important as it dictates and organizes the agent's future behavior and restricts\nwhich potential moves may be considered.</li>\n<li><b>Problem formulation</b>: The agent comes up with a description of the different states in the environment and the actions that are needed to make the goal, i.e.\nan abstract representation of the world.</li>\n<li><b>Search</b>: The agent will simulate multiple different paths to see if it can find a <b>solution</b>, or a sequence of actions that lead to goal state, if one exists.\n</li><li><b>Execution</b>: The agent evaluates the actions from its solution, one at a time.</li></ul>\nThe environment can be further characterized as <b>open-loop</b> or <b>closed-loop</b> if the environment is nondeterministic.\n<br /><br />\nA search <b>problem</b> is formally defined as having the following:\n<ul><li><b>State space</b>: the set of all possible <b>states</b> of the environment.</li>\n<li><b>Initial state</b>: the initial state that the agent is initially in</li>\n<li><b>Goal state(s)</b>: the target state that the agent attempts to reach, can be multiple. A method e.g. <i>Is-Goal(s)</i> should return boolean indicating if reached.</li>\n<li><b>Action(<i>s</i>)</b>: a method which returns a finite set of actions that can be executed given a state <i>s</i></li>\n<li><b>Transition model</b>: a method which denotes the result of each action, e.g. <i>Results(s,a)</i> returns the result of applying action <i>a</i> to state <i>s</i>.</li>\n<li><b>Action cost function</b> <i>c(s,a,s')</i> which denotes the numeric cost of applying an action <i>a</i> on state <i>s</i> to achieve state <i>s'</i>.\nThis cost function should reflect the agent's own performance measure.</li></ul>\nAny sequence of actions leads forms a <b>path</b>, a path that leads to the goal state is a <b>solution</b> and an <b>optimal solution</b> is one that minimizes the path cost.\nWe can also visualize the solution in a <b>graph</b> in which edges are actions between the vertices which are states.\n<br /><br />\nOne final thing to consider is that these representations are a <b>model</b> and these have a specific <b>level of abstraction</b> that removes as much information as possible\nwhile ensuring that <i>validity</i> is preserved and that abstract solutions can be carried out for the agent. The process of removing these other details is <b>abstraction</b>.\n\n## AIMA Chapter 3.2 Example Problems\n It is important first to distinguish between a <b>standardized problem</b> and actual <b>real-world problems</b>. Some examples of this include:\n<ul><li><b>State space</b>: the set of all possible cells. For <b>vacuum world</b>, if there are <i>n</i> cells there are <i>n * 2<sup>n</sup></i> states. This can also be a\n(partially filled) Sudoku board, a description of all the tiles in the <b>sliding-tile puzzle</b>, a <b>sobokan board</b>, the real numbers in Knuth's conjecture, or the\nlocation (e.g. an airport) and current time in the <b>route-finding problem</b>. The Sobokan example can have <i>n * n!/(b!(n-b)!)</i> or over 200 trillion possibilities\nfor an 8x8 board, Knuth's conjecture has infinite possible states.</li>\n<li><b>Initial state</b>: Any state in <b>grid world</b> or any arrangement of <b>sliding-tiles</b> (note that a goal can be reached by about half of the states),\nthe number 4 in Knuth's conjecture, or the user's home airport.</li>\n<li><b>Goal state(s)</b>: For <b>vacuum world</b>, the state in which every cell is clean. For <b>sliding-puzzle</b>, any state, but typically that in which the numbers are in\norder, for Knuth's conjecture, the desired positive integer, and for <b>route planning</b> a specified destination city.</li>\n<li><b>Action(<i>s</i>)</b>: These can be <b>absolute</b> movement actions like <i>Upward, Downward</i> or <b>egocentric actions</b> with respect to the agent, like\n<i>Forward</i>, <i>Backward</i>, etc. for the vacuum world. For the <b>sliding-tile puzzle</b>, this involves moving a tile in one of the cardinal directions to an empty\nspace. For Knuth's conjecture, this can be either the square root, floor, or factorial operations. And for the <b>route-finding</b> problem this is taking any flight\nfrom the current location that leaves after the current time and leaves enough time for a within-airport transfer.</li>\n<li><b>Transition model</b>: certain actions like <i>Suck</i> remove dirt from one cell, resulting in a new state, and motion activities leave the vacuum in a different\nposition. The puzzle pieces move into the empty space if possible, creating a new empty space. The transition model for Knuth's conjecture is the mathematicaloperations,\nand for the route planning problem, it will be the new destination city and flight's arrival time.</li>\n<li><b>Action cost function</b> Each action costs 1 for the <b>vacuum world</b>, <b>sliding-tile puzzle</b> and Knuth's conjecture problems. For the <b>route planning</b>\nalgorithm, this can be a combination of cost (monetary), time (either waiting and/or flying), and other factors like seat quality, flight points, etc.</li></ul>\nSome other more complicated problems that can be solved include the <b>traveling salesperson problem (TSP)</b> which is a type of <b>touring problem</b> in which every city on\na map is to be visited once and <b>VLSI layout</b> which groups cells into an ideal <b>cell layout</b> and then designing a <b>channel routing</b> procedure.\n\n\n## AIMA Chapter 3.3 Search Algorithms\nA <b>search algorithm</b> takes as input a search problem and returns either a solution or failure. A <b>search tree</b> is a representation fo the paths\ntaken from the initial state over the state-space graph that reach towards a goal state. Each <b>node</b> in the search tree corresponds to a state in the search tree and\nits edges correspond to actions from that node. From the root node, we <b>expand</b> each node by considering the available ACTION and RESULT functions to see what other\n<b>child node</b> or <b>successor node</b> can be generated from the current node - now the <b>parent node</b>. The nodes in the search tree which have been reached already\nare the <b>frontier</b> and as a node gets added to this it is considered <b>reached</b>. The frontier hence <b>separates</b> two regions of the state-space graph: an\ninterior where every node has been expanded, and the exterior which has not been reached.\n<br /><br />\nOne such search algorithm is <b>best-first search</b> in which the node <i>n</i> that is chosen has the minimum value of an <b>evaluation fucntion</b> <i>f(n)</i>. The\nalgorithm proceeds on each iteration by checking if it is the goal state (and returning if so), or expanding on its neighbors otherwise. Each node is added to the frontier\nif it has not been reached before or if it has a lower path cost than previously. The algorithm returns either a path to the goal or indicates failure.\n<br /><br />\nA <b>node</b> in a tree is represented by a data structure which has four components:\n<ul><li><i>node</i>.STATE: the state the node corresponds with</li>\n<li><i>node</i>.PARENT: the node from which the node was generated</li>\n<li><i>node</i>.ACTION: the action from which the node was reached (from the parent node)</li>\n<li><i>node</i>.PATH-COST: the total cost along path from initial state to the node, synonymous to <i>g(node)</i></li></ul>\nFollowing the PARENT pointers back allow one to recover the entire states and actions leading to a node. Doing this from the goal leads to the solution.\n<br /><br />\nThe <b>frontier</b> needs to support operations to check if it IS-EMPTY, and to POP and ADD elements. Some types of <b>queues</b> that can be used include:\n<ul><li>A <b>priority queue</b> pops the element with minimum cost according to a function <i>f</i></li>\n<li>A <b>FIFO queue</b> pops the node that was added first</li>\n<li>A <b>LIFO queue</b> (or a <b>stack</b>) pops the node that was added most recently</li></ul>\nEvaluate performance on:\n<ul><li><b>Completeness</b>: guaranteed to find solution when there is one, and correctly report failure if not?</li>\n<li><b>Cost optimality</b>: guaranteed to return lowest cost solution?</li>\n<li><b>Time complexity</b>: how long does it take? measured by # of states/actions considered.</li>\n<li><b>Space complexity</b>: how much memory needed to search?</li></ul>\nSearch is <b>graph search</b> if it checks for <b>redundant path</b>, otherwise <b>tree-like search</b>.\n\n\n## AIMA Chapter 3.4 Uninformed Search Strategies\nTypes of uninformed search include:\n<ol><li><b>Breadth-First Search</b>: a strategy in which a root node is expanded, then its successors, and their successors. Complete even on an infinite state space. \nOne optimization is to do an <b>early goal test</b>, i.e. checking if a goal state has been reached as soon as it has been <i>generated</i> \nas opposed to when dequeued as in with a <b>late goal test</b>.</li>\n<li>Dijkstra's algorithm or <b>uniform cost search</b> spreads out in waves of uniform path-cost. It can be implemented as a call to BEST-FIRST-SEARCH with PATH-COST\nas the evaluation function. BFS tests for the goal state only when it expands a node, not when it generates it.</li>\n<li><b>Depth first search</b> is tree-like search, expands node until it cannot anymore, then reverts to state where there is more to explore. In\n<b>backtracking search</b>, even less memory used as it generates one successor at a time, modifying state description instead of allocating memroy for a \nbrand-new state.</li>\n<li><b>Depth-limited search</b> is a version of DFS where we treat anything beyond a depth-limit <i>l</i> and treat nodes at that depth at terminal. A good heuristic\nis the <b>diameter</b> of state-space graph, but this is not always known. <b>Iterative deepening search</b> works by doing depth-limited search for 1, 2, etc. until\na cutoff value is reached.</li>\n<li><b>Bidirectional search</b> expands from the initial state and backwards from the goal state(s). Need to\nreverse the problem and order of visited solutions for backward direction.</i>\n</ol>\n\n<table>\n<tr><td>Criterion</td><td>Breadth-First</td><td>Uniform-Cost</td><td>Depth-First</td><td>Depth-Limited</td><td>Iterative Deepening</td><td>Bidirectional</td></tr>\n<tr><td>Complete?</td><td>Yes<sup>1</sup></td><td>Yes<sup>1,2</sup></td><td>No</td><td>No</td><td>Yes<sup>1</sup></td><td>Yes<sup>1,4</sup></td></tr>\n<tr><td>Optimal cost?</td><td>Yes<sup>3</sup></td><td>Yes</td><td>No</td><td>No</td><td>Yes<sup>3</sup></td><td>Yes<sup>3,4</sup></td></tr>\n<tr><td>Time</td><td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>1+floor(C*/e)</sup>)</i></td><td><i>O(b<sup>m</sup>)</i></td><td><i>O(b<sup>l</sup>)</i></td>\n<td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>d/2</sup>)</i></td></tr>\n<tr><td>Space</td><td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>1+floor(C*/e)</sup>)</i></td><td><i>O(bm)</i></td></td><td><i>O(bl)</i></td></td><td><i>O(bd)</i></td>\n<td><i>O(b<sup>d/2</sup>)</i></td></tr>\n</table>\nwhere <i>b</i> is the branching factor, <i>m</i> is the maximum depth of the search tree, <i>d</i> is the depth of the shallowest solution or <i>m</i> when no solution,\nand <i>l</i> is the depth-limit.<br />\n<sup>1</sup>: complete if <i>b</i> is finite and state space is either finite or has a solution<br />\n<sup>2</sup>: complete if all actions have cost >= <i>e</i> > 0<br />\n<sup>3</sup>: cost-optimal if actions are all identical<br />\n<sup>4</sup>: if both directions are breadth-first or uniform-cost\n\n## Module 3 Lecture Notes\n Your summary here \n\n## Module 3 Definitions\n* **Problem-solving agent** - an agent that can formulate a plan via search\n* **Atomic** - representations of the world that are considered as a whole with no visible internal structure\n* **Informed algorithm** - an algorithm in which an agent has information to estimate its distance to a goal state\n* **Uninformed algorithm** - an algorithm in which an agent has no information to estimate its distance to a goal state\n* **Solution** - a sequence of actions through which an agent can reach its goal state\n* **Open-loop system** - a system in which an agent can ignore the percepts once its solution is determined\n* **Closed-loop system** - a system in which the environment can change, requiring the agent to rely on percepts\n* **State space** - the set of possible states that the environment can be\n* **Initial state** - the initial state of the environment that the agent starts the problem in\n* **Goal states** - states that the agent intends to achieve\n* **Actions** - the choices an agent can make\n* **Transition model** - a formula that describes what an action does\n* **Action cost function** - a function that describes the cost an agent occurs in an action\n* **Path** - a sequence of actions\n* **Optimal solution** - the solution that has the lowest path cost to the agent\n* **Model** - an abstract mathematical representation\n* **Abstraction** - the process of removing details from a representation\n* **Level of abstraction** - the degree to which detail has been omitted for an action\n* **Standardized problem** - an illustrative problem to show an example of problem-solving\n* **Real-world problem** - a problem that arises from actual use case\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WHREsHSSyA-h"
      },
      "source": [
        "#Module 4 - Informed Search"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tuFC8w7CyCns",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "73c84ffa-da46-4333-9326-872b98546574"
      },
      "source": [
        "# Russell and Norvig, AIMA Chapter 3 “Problem Solving by Search” (3.5-3.6)\n",
        "chapter3_5 = \"\"\"<b>Informed search</b> strategies use a domain-specific <b>heuristic function</b> <i>h(n)</i> to find solutions more efficiently.\n",
        "<b>Greedy best-first search</b>: expands the node with the lowest <i>h(n)</i> value - finds a solution, but may ignore options that initially appear to be not optimal.\n",
        "<br /><br />\n",
        "<b>A* Search</b>: Best-first search using <i>f(n) = g(n) + h(n)</i> where <i>g(n)</i> is path cost from initial state to node <i>n</i>.\n",
        "A* relies on having an <b>admissible heuristic</b> and is cost-optimal with a <b>consistent</b> heuristic. However, can expand an exponential number of nodes.\n",
        "<br /><br />\n",
        "If  a <b>satisficing</b> solution is okay, then it is possible to run A* with an \n",
        "<b>inadmissible heuristic</b> that might miss optimal solution but further reduces the number of nodes expanded. An example is the <b>detour index</b>. \n",
        "This idea can be applied to any problem using the <b>weighted A* search</b>\n",
        "where the heuristic is weighted so evaluation function <i>f(n) = g(n) + W * h(n)</i> for some <i>W > 1</i>. If the optimal solution has cost <i>C*</i>, then\n",
        "weighted A* has a solution between <i>C*</i> and <i>W * C*</i>, generally closer to <i>C*</i>. Some types of suboptimal search include <b>bounded\n",
        "suboptimal search</b>, <b>bounded-cost search</b>, <b>unbounded-cost search</b> (e.g. <b>speedy search</b>).\n",
        "<br /><br />\n",
        "Memory-Bounded Search:\n",
        "<ul><li><b>Beam Search</b>: limits size of the frontier, keeping only the <i>k</i> nodes with the best <i>f</i>-scores. Clearly incomplete and suboptimal, but can execute\n",
        "fast because expands fewer nodes, i.e. exploring only optimal portions of contour.</li>\n",
        "<li><b>Iterative-deepening A* Search</b>: uses cutoff of <i>f</i>-cost (<i>g + h</i>) - for each iteration, the cutoff value is the smallest <i>f</i>-cost that exceeds that of\n",
        "the previous iteration. If optimal solution has cost <i>C*</i>, no more than <i>C*</i> iterations.</li>\n",
        "<li><b>Recursive best-first search</b>: mimics best-first search, but instead of continuing indefinitely down a path, uses the <i>f</i>-limit to keep track of\n",
        "the <i>f</i>-value of the best alternative path from any ancestor of a current node. If the current node exceeds this <i>f</i>-limit, thenRBFSt unwinds back to alternative\n",
        "path. As recursion unwinds, RBFS replaces the <i>f</i>-value of cost along the <b>backed-up value</b> or best <i>f</i>-value of its children.</li>\n",
        "<li><b>MA*</b> (Memory-Bounded A*) and <b>SMA*</b> (Simplified MA*) are better when we can determine how much memory is available. SMA* expands\n",
        "A* until memory is full, then drops the leaf node with highest <i>f</i>-value, only regenerating this when all the other paths are worse than the\n",
        "forgotten path. Complete if any reachable solution is less than memory, optimal if any optimal solution is reachable, but can be very inefficient if forced to switch\n",
        "back and forth between many candidates and re-generate tree.</li></ul>\n",
        "<br />\n",
        "For bidirectional Search, need to run in both directions.\"\"\"\n",
        "chapter3_6 = \"\"\"When considering different heuristic functions, important to consider <b>effective branching factor</b>. If the total number of nodes generated by A* is <i>N</i>\n",
        "and solution depth is <i>d</i> then <i>b*</i> is the branching factor that uniform tree of depth <i>d</i> would have to have in order to contain <i>N + 1</i> nodes. Thus,\n",
        "<i>N + 1 = 1 + b* + (b*)^2 + ... + (b*)^d</i>, e.g. a solution at depth 5 after 52 nodes has EBF of 1.92. A given heuristic <i>h</i> reduces the <b>effective depth</b> by a \n",
        "constant <i>k<sub>h</sub></i> compared to true depth, reducing search cost from uninformed <i>O(b<sup>d</sup>)</i> to <i>O(b<sup>d-k<sub>h</sub></sup>)</i>. A value close to 1 is\n",
        "better, so for two heuristics <i>h<sub>1</sub></i> and <i>h<sub>2</sub></i>, it is said <i>h<sub>2</sub></i> <b>dominates</b> <i>h<sub>2</sub></i> if it expands fewer nodes as\n",
        "every node expanded in the better function will be expanded in the other.\n",
        "<br /><br />\n",
        "There are several ways to generate a heuristic:\n",
        "<ol><li>Create a solution for a <b>relaxed problem</b>, e.g. <b>Manhattan distance</b> solves the 8-block puzzle when the condition that squares cannot be moved to non-empty \n",
        "spots is lifted.</li>\n",
        "<li>Derive he solution cost of a <b>subproblem</b> of a problem, e.g. only considering tiles 1-4 in the 8-block puzzle. These states can be cached in a <b>pattern \n",
        "database</b>, then using that subproblem cost as an admissible heuristic. The other subproblem - 5-8 may have some overlapping moves with this subproblem, \n",
        "so the sum of both may overestimate; then, a <b>disjoint pattern database</b> which excludes common moves should be used.</li>\n",
        "<li><b>Precomputation</b> of some optimal path costs. One approach is to pick 10 to 20 <b>landmark points</b> and then for each landmark <i>L</i> and other vertex\n",
        "<i>v</i>, compute and store <i>C*(v,L)</i>, the optimal path from <i>v</i> to <i>L</i>. This creates an efficient but inadmissible heuristic: <i>h<sub>L</sub>(n) =\n",
        "min<sub>L in Landmarks</sub>C*(n,L) + C*(n,goal)</i> since this is either exact (the node goes through a landmark) or overestimates. \n",
        "<br />\n",
        "One can also compute several <b>shortcuts</b>, e.g. from CA to NY. This is still not admissible, but the <b>differential heuristic</b><i>h<sub>L</sub>(n) =\n",
        "C*(n,L) - C*(goal,L)</i> modifies this slightly so the landmark is now somewhere past the goal. Landmarks should be spread out; they can also be picked based on frequency. \n",
        "They work well in real-world situations because roads are often built along commonly travelled paths.</li>\n",
        "<li>Machine Learning algorithms on the <b>metalevel state space</b> - the entire search tree as opposed to just the current state - e.g. a map of is an <b>object-level\n",
        "state space</b>. <b>Metalevel learning</b> algorithms can learn to avoid certain subtrees and thus minimize the <b>total cost</b> using techniques like reinforcement learning -\n",
        "these work better when <b>features</b> relate to state's heuristic value instead of just describing it.</ol>\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module4_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module4_definitions = {\n",
        "\"informed search\" : \"seearch strategy that uses domain-specific hints to find solutions more efficiently than uninformed search\",\n",
        "\"heuristic function\" : \"<i>h(n)</i> is the <i>estimated</i> cost of  the shortest path from <i>n</i>to a goal-state.\",\n",
        "\"admissibility\" : \"an admissible heuristic is one that is optimistic, i.e. never overestimates the coast to reach a goal\",\n",
        "\"consistency\" : \"a heuristic is consistent if for every node <i>n</i>, its successors <i>n'</i> satisfy <i>h(n) <= c(n,a,n') + h(n')</i>\",\n",
        "\"triangle inequality\" : \"one side of a triangle cannot be larger than the other two sides combined\",\n",
        "\"contours\" : \"an outline of points equidistant from ainitial state\",\n",
        "\"monotonic\" : \"the path cost is always increasing along a certain path\",\n",
        "\"surely expanded nodes\" : \"all nodes reachable from an initial state on a path where every node has less than optimal cost\",\n",
        "\"optimally efficient\" : \"an algorithm such that any algorithm that achieves the same state must pass through the same nodes\",\n",
        "\"prunes\" : \"eliminating possible candidates for expansion without having to first examine them\",\n",
        "\"satisficing\" : \"good enough\",\n",
        "\"inadmissible heuristic\" : \"a heuristic that overestimates\",\n",
        "\"detour index\" : \"heuristic which applies a multiplier to the straight-line distance (e.g. 10 mi apart -> estimate 13 mi)\",\n",
        "\"bounded suboptimal search\" : \"a solution that is guaranteed to be within a constant factor <i>W</i> of the optimal cost\"\n",
        "\n",
        "}\n",
        "\n",
        "print(\"The length in words is\", len(chapter3_6.split(' ')))\n",
        "print(\"The length in characters is\", len(chapter3_6))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 428\n",
            "The length in characters is 2999\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vqtEs0gHs9sO",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1330
        },
        "cellView": "form",
        "outputId": "f3cf3589-bbac-4c54-c680-750c272ab33d"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module4_sections = [\n",
        "  (\"AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\", chapter3_5),\n",
        "  (\"AIMA Chapter 3.6 Heuristic Functions\", chapter3_6),\n",
        "  (\"Module 4 Lecture Notes\", module4_lecture_notes),\n",
        "]\n",
        "\n",
        "module4_markdown = format_module(4, \"Informed Search\", \n",
        "                         module4_sections, \n",
        "                         module4_definitions)\n",
        "display(Markdown(module4_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 4: Informed Search\n## AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\n<b>Informed search</b> strategies use a domain-specific <b>heuristic function</b> <i>h(n)</i> to find solutions more efficiently.\n<b>Greedy best-first search</b>: expands the node with the lowest <i>h(n)</i> value - finds a solution, but may ignore options that initially appear to be not optimal.\n<br /><br />\n<b>A* Search</b>: Best-first search using <i>f(n) = g(n) + h(n)</i> where <i>g(n)</i> is path cost from initial state to node <i>n</i>.\nA* relies on having an <b>admissible heuristic</b> and is cost-optimal with a <b>consistent</b> heuristic. However, can expand an exponential number of nodes.\n<br /><br />\nIf  a <b>satisficing</b> solution is okay, then it is possible to run A* with an \n<b>inadmissible heuristic</b> that might miss optimal solution but further reduces the number of nodes expanded. An example is the <b>detour index</b>. \nThis idea can be applied to any problem using the <b>weighted A* search</b>\nwhere the heuristic is weighted so evaluation function <i>f(n) = g(n) + W * h(n)</i> for some <i>W > 1</i>. If the optimal solution has cost <i>C*</i>, then\nweighted A* has a solution between <i>C*</i> and <i>W * C*</i>, generally closer to <i>C*</i>. Some types of suboptimal search include <b>bounded\nsuboptimal search</b>, <b>bounded-cost search</b>, <b>unbounded-cost search</b> (e.g. <b>speedy search</b>).\n<br /><br />\nMemory-Bounded Search:\n<ul><li><b>Beam Search</b>: limits size of the frontier, keeping only the <i>k</i> nodes with the best <i>f</i>-scores. Clearly incomplete and suboptimal, but can execute\nfast because expands fewer nodes, i.e. exploring only optimal portions of contour.</li>\n<li><b>Iterative-deepening A* Search</b>: uses cutoff of <i>f</i>-cost (<i>g + h</i>) - for each iteration, the cutoff value is the smallest <i>f</i>-cost that exceeds that of\nthe previous iteration. If optimal solution has cost <i>C*</i>, no more than <i>C*</i> iterations.</li>\n<li><b>Recursive best-first search</b>: mimics best-first search, but instead of continuing indefinitely down a path, uses the <i>f</i>-limit to keep track of\nthe <i>f</i>-value of the best alternative path from any ancestor of a current node. If the current node exceeds this <i>f</i>-limit, thenRBFSt unwinds back to alternative\npath. As recursion unwinds, RBFS replaces the <i>f</i>-value of cost along the <b>backed-up value</b> or best <i>f</i>-value of its children.</li>\n<li><b>MA*</b> (Memory-Bounded A*) and <b>SMA*</b> (Simplified MA*) are better when we can determine how much memory is available. SMA* expands\nA* until memory is full, then drops the leaf node with highest <i>f</i>-value, only regenerating this when all the other paths are worse than the\nforgotten path. Complete if any reachable solution is less than memory, optimal if any optimal solution is reachable, but can be very inefficient if forced to switch\nback and forth between many candidates and re-generate tree.</li></ul>\n<br />\nFor bidirectional Search, need to run in both directions.\n\n## AIMA Chapter 3.6 Heuristic Functions\nWhen considering different heuristic functions, important to consider <b>effective branching factor</b>. If the total number of nodes generated by A* is <i>N</i>\nand solution depth is <i>d</i> then <i>b*</i> is the branching factor that uniform tree of depth <i>d</i> would have to have in order to contain <i>N + 1</i> nodes. Thus,\n<i>N + 1 = 1 + b* + (b*)^2 + ... + (b*)^d</i>, e.g. a solution at depth 5 after 52 nodes has EBF of 1.92. A given heuristic <i>h</i> reduces the <b>effective depth</b> by a \nconstant <i>k<sub>h</sub></i> compared to true depth, reducing search cost from uninformed <i>O(b<sup>d</sup>)</i> to <i>O(b<sup>d-k<sub>h</sub></sup>)</i>. A value close to 1 is\nbetter, so for two heuristics <i>h<sub>1</sub></i> and <i>h<sub>2</sub></i>, it is said <i>h<sub>2</sub></i> <b>dominates</b> <i>h<sub>2</sub></i> if it expands fewer nodes as\nevery node expanded in the better function will be expanded in the other.\n<br /><br />\nThere are several ways to generate a heuristic:\n<ol><li>Create a solution for a <b>relaxed problem</b>, e.g. <b>Manhattan distance</b> solves the 8-block puzzle when the condition that squares cannot be moved to non-empty \nspots is lifted.</li>\n<li>Derive he solution cost of a <b>subproblem</b> of a problem, e.g. only considering tiles 1-4 in the 8-block puzzle. These states can be cached in a <b>pattern \ndatabase</b>, then using that subproblem cost as an admissible heuristic. The other subproblem - 5-8 may have some overlapping moves with this subproblem, \nso the sum of both may overestimate; then, a <b>disjoint pattern database</b> which excludes common moves should be used.</li>\n<li><b>Precomputation</b> of some optimal path costs. One approach is to pick 10 to 20 <b>landmark points</b> and then for each landmark <i>L</i> and other vertex\n<i>v</i>, compute and store <i>C*(v,L)</i>, the optimal path from <i>v</i> to <i>L</i>. This creates an efficient but inadmissible heuristic: <i>h<sub>L</sub>(n) =\nmin<sub>L in Landmarks</sub>C*(n,L) + C*(n,goal)</i> since this is either exact (the node goes through a landmark) or overestimates. \n<br />\nOne can also compute several <b>shortcuts</b>, e.g. from CA to NY. This is still not admissible, but the <b>differential heuristic</b><i>h<sub>L</sub>(n) =\nC*(n,L) - C*(goal,L)</i> modifies this slightly so the landmark is now somewhere past the goal. Landmarks should be spread out; they can also be picked based on frequency. \nThey work well in real-world situations because roads are often built along commonly travelled paths.</li>\n<li>Machine Learning algorithms on the <b>metalevel state space</b> - the entire search tree as opposed to just the current state - e.g. a map of is an <b>object-level\nstate space</b>. <b>Metalevel learning</b> algorithms can learn to avoid certain subtrees and thus minimize the <b>total cost</b> using techniques like reinforcement learning -\nthese work better when <b>features</b> relate to state's heuristic value instead of just describing it.</ol>\n\n\n## Module 4 Lecture Notes\n Your summary here \n\n## Module 4 Definitions\n* **Informed search** - seearch strategy that uses domain-specific hints to find solutions more efficiently than uninformed search\n* **Heuristic function** - <i>h(n)</i> is the <i>estimated</i> cost of  the shortest path from <i>n</i>to a goal-state.\n* **Admissibility** - an admissible heuristic is one that is optimistic, i.e. never overestimates the coast to reach a goal\n* **Consistency** - a heuristic is consistent if for every node <i>n</i>, its successors <i>n'</i> satisfy <i>h(n) <= c(n,a,n') + h(n')</i>\n* **Triangle inequality** - one side of a triangle cannot be larger than the other two sides combined\n* **Contours** - an outline of points equidistant from ainitial state\n* **Monotonic** - the path cost is always increasing along a certain path\n* **Surely expanded nodes** - all nodes reachable from an initial state on a path where every node has less than optimal cost\n* **Optimally efficient** - an algorithm such that any algorithm that achieves the same state must pass through the same nodes\n* **Prunes** - eliminating possible candidates for expansion without having to first examine them\n* **Satisficing** - good enough\n* **Inadmissible heuristic** - a heuristic that overestimates\n* **Detour index** - heuristic which applies a multiplier to the straight-line distance (e.g. 10 mi apart -> estimate 13 mi)\n* **Bounded suboptimal search** - a solution that is guaranteed to be within a constant factor <i>W</i> of the optimal cost\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F3FPMzG9yUls"
      },
      "source": [
        "#Module 5 - Adversarial Search and Games"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iEIOpL7vyWcc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3c3137dd-184f-455a-e5f6-0b2761c6bf42"
      },
      "source": [
        "# Russell and Norvig, AIMA chapter 5 \"Adversarial Search\"”\" (5.1-5.3, and 5.5).\n",
        "chapter5_1 = \"\"\"In game theory, there are 3 approaches: \n",
        "<ol><li>The behaviors of a large number of agents can be viewed in aggregate as an <b>economy</b>.</li>\n",
        "<li>Adversarial agents are part of the environment, which is nondeterministic.</li>\n",
        "<li>Explicitly model the adversarial agents with techniques like adversarial game-tree search.</li></ol>\n",
        "Here we focus on the 3rd and use an algorithm to define the optimal move via minimax search, which is an optimization of <i>AND-OR</i> search.\n",
        "Further techniques like <b>pruning</b> which leads the algorithm to ignore portions of the search tree that will not result in an optimal move.\n",
        "In order to determine which method is winning, we apply a heuristic <b>evaluation function</b> to determine which agent is winning based on the current state of the board.\n",
        "<br />\n",
        "Many of the games studied within AI are deterministic, two-player, turn-taking, <b>perfect information</b> (fully observable), <b>zero-sum games</b> (what is good for one\n",
        "player is bad for the other). In this context, <b>move</b> is synonymous with \"action\" and <b>position</b> is synonymous with \"state\". Here, the two players are called \n",
        "<i>MAX</i> and <i>MIN</i>, where <i>MAX</i> moves first, and the players take turns until the game reaches a terminal state. At the end, the winning party gets the points \n",
        "and the other party is given a penalty. The game can be formally defined with the following elements:\n",
        "<ul><li>The <b>initial state</b> is the state of the game when it is first initialized</li>\n",
        "<li><i>TO-MOVE(s)</i>: Returns which agent's turn it is to move at state <i>s</i>.</li>\n",
        "<li><i>ACTION(s)</i>: Returns the set of all legal moves in <i>s</i></li>\n",
        "<li><i>RESULT(s, a)</i>: The <b>transition model</b>, which is the state of the game after action <i>a</i> is taken in state <i>s</i>.</li>\n",
        "<li><i>IS-TERMINAL(s)</i>: Returns the result of the <b>terminal test</b>, which is true when the game is over (and otherwise false). States in which the game passes the\n",
        "terminal test are called <b>terminal states</b>.</li>\n",
        "<li><i>UTILITY(s, p)</i>: Returns the result of the <b>utility function</b> (otherwise known as the objective or payoff function), which assigns a numeric value to player\n",
        "<i>p</i> for the game ending in the terminal state <i>s</i> For chess, a win is defined as 1 point, a loss as 0 points, and a draw as 1/2 point. In other games like\n",
        "backgammon the outcomes can be in the range of 0 to 192.</li></ul>\n",
        "The initial state, the <i>ACTION</i>, and <i>RESULT</i> functions define the <b>state space graph</b>, which is a graph where the edges are moves, the vertices are states, and\n",
        "a state can be reached by multiple moves. A <b>search tree</b> is superimposed over part of this graph to determine what is the best move to make. A <b>game tree</b> is a search \n",
        "tree which follows every sequence of moves leading up to a terminal state. The game tree can be infinite if the game allows for infinitely many repeating positions or if \n",
        "the state space graph is unbounded.\"\"\"\n",
        "chapter5_2 = \"\"\"A modified version of <i>AND-OR</i> search called  <b>minimax search</b> models this behavior. Here, a turn taken by one player is a <b>ply</b>, and at each\n",
        "layer the goal is the <b>minimax value</b> for each state of the tree, denoted by <i>MINIMAX(s)</i>. For a terminal state, this is its utility. <i>MAX</i> prefers\n",
        "a move to a state of maximum value while <i>MIN</i> prefers a move to a state of minimum value. This results in the formula:\n",
        "<table>\n",
        "<tr><td><i>MINIMAX(s)</i>=</td><td></td></tr>\n",
        "<tr><td><i>UTILITY(s,MAX)</i></td><td>if <i>IS-TERMINAL(s)</i></td></tr>\n",
        "<tr><td><i>max<sub>a in Action(s)</sub> MINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n",
        "<tr><td><i>max<sub>a in Action(s)</sub> MINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n",
        "</table>\n",
        "<br />\n",
        "e.g. a first <i>MIN</i> node has values 3, 12, and 8, so the minimax value is 3. The other successor states of the root have minimax values of 2,\n",
        "so the root has a minimax value of 3. The root is the <b>minimax decision</b> or optimal choice for <i>MAX</i> because it leads to the state with highest minimax value. The\n",
        "minimax algorithm works by computing the best move for <i>MAX</i> by choosing the state with the highest <i>MINIMAX</i> value. It recursively proceeds all the way down to\n",
        "the leaves and then <b>backs up</b> the minimax values as the recursion unwinds. The time complexity is <i>O(b<sup>m</sup>)</i> and the space complexity is <i>O(bm)</i> when all\n",
        "actions are generated at once or <i>O(m)</i> when generated one at a time.\n",
        "<br /><br />\n",
        "In multiplayer games, players form <b>alliances</b>, formally or informally, e.g. players A and B target C if C appears weaker.\n",
        "<br /><br />\n",
        "To reduce the number of game states, <b>pruning</b> large parts of the tree that make no difference to the outcome can be done as in <b>alpha-beta pruning</b>. This involves\n",
        "passing additional parameters to <i>MAX_VALUE(state, alpha, beta)</i> where\n",
        "<ul><li><i>alpha</i> = the value of the best (i.e. highest-value) choice along the path for <i>MAX</i> (think: \"at least\")</li>\n",
        "<li><i>beta</i> = the value of the best (i.e. lwoest-value) choice along the path for <i>MIN</i> (think: \"at most\")</li></ul>\n",
        "The branch factor can become <i>b<sup>0.5</sup></i> instead of <i>b</i>, so we only need to examine <i>O(b<sup>m/2</sup>)</i> instead of <i>O(b<sup>m</sup>)</i>\"\n",
        "nodes. Some other considerations include <b>iterative deepening</b> to expand a level before moving on to next, focusing on <b>killer moves</b> and try them first and to\n",
        "consider using a <b>transposition table</b> that caches the heuristic value of states in the case of <b>transpositions</b> or permutations that result in the same game state.\n",
        "<b>Type A strategy</b> considers all possible moves to a certain depth and evaluates them using a heuristic while a <b>Type B Strategy</b> ignores moves that look bad,\n",
        "instead focusing on a <i>deep but narrow</i> portion of the tree.\"\"\"\n",
        "chapter5_3 = \"\"\"Because computation time is limited, we cut off search early and use a heuristic <b>evaluation function</b> on the states, effectively treating them as terminal.\n",
        "We replace the <i>UTILITY</i> function with the <i>EVAL</i> function, and also replace the terminal test with a <b>cutoff test</b> - which returns <i>True</i> for terminal\n",
        "states or states past the cutoff depth. This gives the formula:\n",
        "<table>\n",
        "<tr><td><i>H-MINIMAX(s)</i>=</td><td></td></tr>\n",
        "<tr><td><i>EVAL(s,MAX)</i></td><td>if <i>IS-CUTOFF(s)</i></td></tr>\n",
        "<tr><td><i>max<sub>a in Action(s)</sub> H-MINIMAX(RESULT(s,a),d+1)</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n",
        "<tr><td><i>max<sub>a in Action(s)</sub> H-MINIMAX(RESULT(s,a),d+1)</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n",
        "</table>\n",
        "<br />\n",
        "For terminal states, <i>EVAL(s,p)=UTILITY(s,p)</i> and for nonterminal states, <i>UTILITY(loss,p)<=EVAL(s,p)<=UTILITY(win,p)</i>. Most evaluation functions work by taking on \n",
        "the <b>expected value</b> various <b>features</b> of the multiple <i>categories</i> or <i>equivalence classes</i>  and the <i>proportion</i> of each states with each outcome.\n",
        "A simple example is taking on the <b>material value</b> of each chess piece and then taking a <b>weighted linear function</b>, e.g. <i>EVAL(s) = w<sub>1</sub>f<sub>1</sub>(s) +\n",
        "... + w<sub>n</sub>f<sub>n</sub>(s)</i>. Some further modifications include applying the evaluation function only to <b>quiescent</b> positions where there is no pending move\n",
        "which swings the evaluation, and doing an extra <b>quiescence search</b> otherwise. There is also the <b>horizon effect</b> in which certain moves can be pushed off until\n",
        "later, which can be mitigated by <b>singular extensions</b>, moves that are clearly better than others.\n",
        "<br /><br />\n",
        "Another issue is with alpha-beta pruning potentially <b>forward pruning</b> some states that appear poor but are better. Some alternatives include <b>beam search</b> which\n",
        "only includes the top <i>n</i> candidates but is similarly problematic, or methods like <i>PROBCUT</i> which considers moves <i>probably</i> instead of <i>provably</i> outside\n",
        "the <i>(alpha, beta)</i> window. Another technique called <b>late move reduction</b> assumes moves are selected well, and discards moves that are later on. More complicated\n",
        "algorithms like <i>STOCKFISH</i> also exist.\n",
        "<br /><br />\n",
        "For openings and closings, a program can be trained to use a <i>table lookup</i> instead of calculating specific states. These include common chess openings. The table\n",
        "can also be constructed by <b>retrogrdae</b> minimax search, which works backwards to identify particularly effective states.\n",
        "\"\"\"\n",
        "chapter5_5 = \"\"\"<b>Stochastic games</b> introduce an element of chance by introducing random elements, e.g. throwing a dice or drawing a card. An example of this is Backgammon.\n",
        "Because Black cannot know with certainty what outcome White will draw, the standard game tree from chess/tic-tac-toe cannot be used, but instead a game tree for Backgammon must\n",
        "include <b>chance nodes</b> in addition to the <i>MAX</i> and <i>MIN</i> nodes. The branches leading from each chance node represent the probability for each dice roll: while\n",
        "there are 36 possible ways to roll two dice, each with 1/36 probability, order does not matter, i.e. a 4-2 is the same a s a 2-4. Therefore, there are only six distinct rolls.\n",
        "The six doubles (e.g. 1-1,2-2,...6-6) each have probability of 1/36, e.g. P(3-3)=1/36, while the other 15 outcomes have a probability of 1/18.\n",
        "<br /><br />Now, since the goal is still to make the correct decision but the positions do not have definite values, we instead calculate the <b>expected value</b> of a\n",
        "position, which is the average of all the possible outcomes over the chance nodes. This leads to the <b>expectiminimax value</b> for stochastic games, which is a generalization\n",
        "of the minimax value used with deterministic nodes. Terminal nodes for <i>MAX</i> and <i>MIN</i> work the same way as before. For chance nodes, the expected value which is the\n",
        "sum of the value over all outcomes, weighted by the likelihood or probability of each action:\n",
        "<table>\n",
        "<tr><td><i>EXPECTIMAX(s)</i>=</td><td></td></tr>\n",
        "<tr><td><i>UTILITY(s,MAX)</i></td><td>if <i>IS-TERMINAL(s)</i></td></tr>\n",
        "<tr><td><i>max(a) EXPECTIMINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n",
        "<tr><td><i>min(a) EXPECTIMINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n",
        "<tr><td><i>sum(r) P(r) * EXPECTIMINIMAX(RESULT(s,r))</i></td><td> if <i>TO-MOVE(s)=CHANCE</i></td></tr>\n",
        "</table>\n",
        "Here, <i>r</i> represents a possible dice roll (or other random event) and <i>RESULT(s, r)</i> is the state of <i>s</i> with the additional fact that the dice roll has\n",
        "resulted in <i>r</i>.<br /><br />Now, with approximate values, the evaluation functions do not just need to give higher values to better positions, but they need to be more\n",
        "careful - the program behaves differently if a change is made to some of evaluation values even if preference remains the same, e.g. [2,3,4,5] vs [20,300,400,5000]. Hence,\n",
        "instead, the evaluation function returns values that are a positive linear transformation of the expected utility - for games with outcomes of win/lose this is the \n",
        "<b>probability</b>  of winning.<br /><br />Now, if minimax can run in <i>O(b<sup>m</sup>)</i> time, where <i>b</i> is the branching factor and <i>m</i>is the maximum depth\n",
        "of the tree, then expectimax must always take into account the number <i>n</i> of distinct roles, resulting in a runtime of <i>O(b<sup>m</sup>n<sup>m</sup>)</i>. This makes it\n",
        "very impractical to run more than a few ply of a game like Backgammon.\"\"\"\n",
        "\n",
        "# Russell and Norvig, AIMA Chapter 16 “Making Simple Decisions” (16.1-16.3)\n",
        "chapter16_1 = \"\"\"Given a state <i>s</i>, we can say that the state has a probability associated with it that can be expressed as <i>P(s)</i>. Then for each action <i>a</i> that\n",
        "an agent can tak, the resulting state and the uncertainty of it can be expressed by <i>P(s'|s,a)</i>, i.e. the probability that the action <i>a</i> in state <i>s</i> leads to\n",
        "state <i>s'</i>. Alternatively, because the focus is the agent reaching state <i>s'</i>, the notation <i>P(RESULT(a)=s')</i> can be used. This is related to all the current\n",
        "states by the formula \n",
        "<br /><i>P(RESULT(a)=s') = sum<sub>s</sub>P(s)P(s'|s,a)</i><br />\n",
        "Decision theory concerns itself with finding the choice of action that has the best <i>immediate</i> result. In order to calculate this, the agent uses the <b>utility\n",
        "function</b> <i>U(s)</i> across all the potential states resulting from an action to calculate the <b>expected utility</b> <i>EU(a)</i>, which is also equivalent to\n",
        "the average utility value of the outcomes weighted by the probability of each outcome occuring, i.e.\n",
        "<br /><i>EU(a) = sum<sub>s'</sub>P(RESULT(a)=s')U(s')</i><br />\n",
        "The general principle that is followed is that of <b>maximum expected utility (MEU)</b> which states that a rational agent should seek to choose the action that provides\n",
        "maximum utility, i.e.\n",
        "<br /><i>action = argmax<sub>a</sub>EU(a)</i><br />\n",
        "This principle <i>formalizes</i> the general notion that a rational agent should do the optimal thing but does not <i>operationalzie</i> the action that does. Obtaining\n",
        "a good utility function and perceiving the world is still a challenge and the agent must also be able to make the right actions based on the values it obtains from this.\n",
        "This results in the somewhat tautological view that the agent that acts to maximize its utility function should correctly reflect its performance measure, resulting in\n",
        "the agent obtaining the highest possible score when weighted against all of the potential environments. This correlates the external performance with its internal\n",
        "representation.\"\"\"\n",
        "chapter16_2 = \"\"\"\n",
        "We can express an agent's preferences with the following notation:\n",
        "<table>\n",
        "<tr><td><i>A > B</i></td><td>the agent prefers <i>A</i> instead of <i>B</i></td></tr>\n",
        "<tr><td><i>A ~ B</i></td><td>the agent is indifferent between <i>A</i> and <i>B</i></td></tr>\n",
        "</table>\n",
        "For a given action, can consider the set of possible outcomes each with respective probability of occuring a <b>lottery</b> <i>L</i> such that:\n",
        "<br /><i>L = [p<sub>1</sub>,S<sub>1</sub>;p<sub>2</sub>,S<sub>2</sub>;...;p<sub>n</sub>,S<sub>n</sub>]</i>\n",
        "Each outcome <i>S<sub>i</sub></i> can either be an atomic sentence or another lottery. There are six main axioms of utility theory:\n",
        "<ol><li><b>Orderability</b>: Exactly one of <i>A > B</i>, <i>B > A</i> or <i>A ~ B </i> holds.</li>\n",
        "<li><b>Transitivity</b>: <i>(A > B) /\\ (B > C) -> (A > C)</i></li>\n",
        "<li><b>Continuity</b>: If a lottery <i>B</i> is in between <i>A</i> and <i>C</i> in preference, then there is some probability <i>p</i> for which agent will be indifferent\n",
        "between getting <i>B</i> in a lottery that yields <i>A</i> with probability <i>p</i> and <i>C</i> with probability <i>1-p</i>:\n",
        "<br /><i>A > B > C -> exists(p) [p;A;1-p,C] ~ B</i></li>\n",
        "<li><b>Substitutability</b>: Given two lotteries, w.l.o.g. <i>A</i> and <i>B</i>, then the agent is indifferent between two or more complex\n",
        "lotteries such that the only difference is that <i>B</i> is substituted for <i>A</i> in one of them, i.e.\n",
        "<br /><i>A ~ b -> [p,A;1-p,C] ~ [p,B;1-p,C]</i></li>\n",
        "<li><b>Monotonicity</b>: Given two lotteries with the same two possible outcomes, w.l.o.g. <i>A</i> and <i>B</i>, if an agent prefers <i>A</i> to <i>B</i>,\n",
        "the agent also prefers the lottery that has a higher probability of <i>A</i> (and vice versa):\n",
        "<br /><i>A > B -> (p > Q <-> [p,A;1-p,B] > [q,A;1-q,B])</i></li>\n",
        "<li><b>Decomposability</b>:Compound lotteries can be reduced to simpler ones using various laws of probability; for example, two consecutive lotteries can be condensed into\n",
        "a single lottery as follows:<br /><i>[p,A;1-p,[q,B;1-q,C]] ~ [p,A;(1-p)q,B;(1-p)(1-q),C]</i></li>\n",
        "</ol>\n",
        "These axioms of utility theory are:\n",
        "<ul>\n",
        "<li><b>Existence of utility Function</b>: if an agent has preferences that obey the axioms of utility, it follows that there exists a function <i>U</i> such that\n",
        "<i>(U(A) > U(B)) <-> (A > B)</i> and <i>(U(A) = U(B)) <-> (A ~ B)</i></li>\n",
        "<li><b>Expected Utility of a Lottery</b>: the sum of the probability of each outcome multipled by the utility of each outcome, i.e.\n",
        "<br /><i>U([p<sub>1</sub>,S<sub>1</sub>;...;p<sub>n</sub>,S<sub>n</sub>] = sum<sub>i</sub>p<sub>i</sub>U(S<sub>i</sub>)</i></li>\n",
        "</ul>\n",
        "These theorems state that a utility function <i>exists</i> but not necessarily that an <i>unique</i> one exists. It is easy to see that any linear transformation, e.g.\n",
        "<i>U'(S) = aU(S) + b</i> is still valid. An agent only needs a preference ranking on these states not the actual values. This is called a <b>value function</b>\n",
        "or an <b>ordinal utility function</b>\"\"\"\n",
        "chapter16_3 = \"\"\"One shortcoming of utility functions is they do not account for individual's preferences, which can lead to behavior that may initially appear irrational\n",
        "but is substnatiated. These can be realized through a process called <b>preference elicitation</b>, whereby an agent is presented with different choices and then asked\n",
        "what their preferences are.\n",
        "<br /><br />\n",
        "It is flawed to assume people have a <b>monotonic preference</b> for more money; given a gamble to wager 1 million for 2.5 by a coin flip, most people would not as the\n",
        "<b>expected money value</b> is not equal. For very rich or very poor people, they are more likely to gamble, indicating that the EMV of money is possibly logarithmic:\n",
        "people at the curves are <b>risk-averse</b> while those at the fringes are <b>risk-seeking</b>.\n",
        "<br /><br />\n",
        "We also do not perceive the world perfectly and are calculating the true utility. In reality, we are calculating an estimate of the utility function\n",
        "<i>EU<sub>est</sub>(a)</i> and in order for this to be <b>unbiased</b> the error must be 0, i.e. <i>E(EU<sub>est</sub>(a) - EU(a)) = 0</i>. We can often calculate the maximum\n",
        "of <i>k</i> estimates and then end up calculating an <b>order statistic</b> overestimate, for <i>X*</i> being the largest in a set {<i>X<sub>1</sub>,...X<sub>k</sub></i>},\n",
        "<br /><i>P</i>(max{<i>X<sub>1</sub>,...X<sub>k</sub></i>} <= <i>x</i>) = <i>F(x)<sup>k</sup></i>\n",
        "<br />and by its derivative,\n",
        "<br /><i>P(x) = kf(x)(F(x))<sup>k-1</sup></i>.\n",
        "<br /><br />\n",
        "The <b>optimizer's curse</b> states that the estimated expected choice is likely to be too high. A study that shows a vaccine is 90% effective is not likely to be 90% effective on\n",
        "future victims for example, and most times something that is effective on 9/10 patients in fact performs worse than something else which is effective on 8000/10000\n",
        "patients. We can aim to mitigate this with a Bayesian approach that uses an explicit probability model <b>P</b>(<i>EU<sub>est</sub>|EU</i>) of the error in utility rates.\n",
        "<br /><br />\n",
        "Some phenomenon that can be observed include:\n",
        "<ul><li>the <b>certainty effect</b>: people are more likely to choose certain outcomes over ambiguous outcomes that may have a higher utility value (even in expectation).\n",
        "People can be described as having <b>ambiguity aversion</b></li>\n",
        "<li>the <b>framing effect</b>: people are more likely to respond positively to a large positive number than a small negative number, e.g. a 80% chance of living is seen\n",
        "as better than a 20% chance of dying even though those are the same.</li>\n",
        "<li>the <b>anchoring effect</b>: people judge based on relative not absolute utility. Pricing an expensive item like a 300 dollar bottle of wine or an overpriced medium box of\n",
        "popcorn drives people to get the 55 dollar bottle or the large popcorn.</li>\n",
        "Investigation into phenomena like this from the perspective of <b>evolutionary psychology</b> indicates that the brain's inherent decision making systems may not view the\n",
        "world through probabilities.\"\"\"\n",
        "\n",
        "\n",
        "print(\"The length in words is\", len(chapter16_2.split(' ')))\n",
        "print(\"The length in characters is\", len(chapter16_2))\n",
        "\n",
        "# Lecture notes \n",
        "module5_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "# Definitions\n",
        "module5_definitions = {\n",
        "\"transition model\" : \"the resulting state after an action is taken\",\n",
        "\"utility function\" : \"function for the numeric value assigned to a player when the game terminates\",\n",
        "\"expected utility\" : \"the average value of the utility function over all the potential outcomes\", \n",
        "\"maximum expected utility\" : \"the principle that a rational agent should choose actions that maximize expected utility\" \n",
        "\n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 383\n",
            "The length in characters is 2973\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "KUc_LgzDvWf-",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 3575
        },
        "outputId": "29ca03ca-dd21-4d45-a639-52d129d7d9d3"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module5_sections = [\n",
        "  (\"AIMA Chapter 5.1 Game Theory\", chapter5_1),\n",
        "  (\"AIMA Chapter 5.2 Optimal Decisions in Games\", chapter5_2),\n",
        "  (\"AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\", chapter5_3),\n",
        "  (\"AIMA Chapter 5.5 Stochastic Games\", chapter5_5),\n",
        "  (\"AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\", chapter16_1),\n",
        "  (\"AIMA Chapter 16.2 The Basis of Utility Theory\", chapter16_2),\n",
        "  (\"AIMA Chapter 16.3 Utility Functions\", chapter16_3),\n",
        "  (\"Module 5 Lecture Notes\", module5_lecture_notes),\n",
        "]\n",
        "\n",
        "module5_markdown = format_module(5, \"Adversarial Search and Games\", \n",
        "                         module5_sections, \n",
        "                         module5_definitions)\n",
        "display(Markdown(module5_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 5: Adversarial Search and Games\n## AIMA Chapter 5.1 Game Theory\nIn game theory, there are 3 approaches: \n<ol><li>The behaviors of a large number of agents can be viewed in aggregate as an <b>economy</b>.</li>\n<li>Adversarial agents are part of the environment, which is nondeterministic.</li>\n<li>Explicitly model the adversarial agents with techniques like adversarial game-tree search.</li></ol>\nHere we focus on the 3rd and use an algorithm to define the optimal move via minimax search, which is an optimization of <i>AND-OR</i> search.\nFurther techniques like <b>pruning</b> which leads the algorithm to ignore portions of the search tree that will not result in an optimal move.\nIn order to determine which method is winning, we apply a heuristic <b>evaluation function</b> to determine which agent is winning based on the current state of the board.\n<br />\nMany of the games studied within AI are deterministic, two-player, turn-taking, <b>perfect information</b> (fully observable), <b>zero-sum games</b> (what is good for one\nplayer is bad for the other). In this context, <b>move</b> is synonymous with \"action\" and <b>position</b> is synonymous with \"state\". Here, the two players are called \n<i>MAX</i> and <i>MIN</i>, where <i>MAX</i> moves first, and the players take turns until the game reaches a terminal state. At the end, the winning party gets the points \nand the other party is given a penalty. The game can be formally defined with the following elements:\n<ul><li>The <b>initial state</b> is the state of the game when it is first initialized</li>\n<li><i>TO-MOVE(s)</i>: Returns which agent's turn it is to move at state <i>s</i>.</li>\n<li><i>ACTION(s)</i>: Returns the set of all legal moves in <i>s</i></li>\n<li><i>RESULT(s, a)</i>: The <b>transition model</b>, which is the state of the game after action <i>a</i> is taken in state <i>s</i>.</li>\n<li><i>IS-TERMINAL(s)</i>: Returns the result of the <b>terminal test</b>, which is true when the game is over (and otherwise false). States in which the game passes the\nterminal test are called <b>terminal states</b>.</li>\n<li><i>UTILITY(s, p)</i>: Returns the result of the <b>utility function</b> (otherwise known as the objective or payoff function), which assigns a numeric value to player\n<i>p</i> for the game ending in the terminal state <i>s</i> For chess, a win is defined as 1 point, a loss as 0 points, and a draw as 1/2 point. In other games like\nbackgammon the outcomes can be in the range of 0 to 192.</li></ul>\nThe initial state, the <i>ACTION</i>, and <i>RESULT</i> functions define the <b>state space graph</b>, which is a graph where the edges are moves, the vertices are states, and\na state can be reached by multiple moves. A <b>search tree</b> is superimposed over part of this graph to determine what is the best move to make. A <b>game tree</b> is a search \ntree which follows every sequence of moves leading up to a terminal state. The game tree can be infinite if the game allows for infinitely many repeating positions or if \nthe state space graph is unbounded.\n\n## AIMA Chapter 5.2 Optimal Decisions in Games\nA modified version of <i>AND-OR</i> search called  <b>minimax search</b> models this behavior. Here, a turn taken by one player is a <b>ply</b>, and at each\nlayer the goal is the <b>minimax value</b> for each state of the tree, denoted by <i>MINIMAX(s)</i>. For a terminal state, this is its utility. <i>MAX</i> prefers\na move to a state of maximum value while <i>MIN</i> prefers a move to a state of minimum value. This results in the formula:\n<table>\n<tr><td><i>MINIMAX(s)</i>=</td><td></td></tr>\n<tr><td><i>UTILITY(s,MAX)</i></td><td>if <i>IS-TERMINAL(s)</i></td></tr>\n<tr><td><i>max<sub>a in Action(s)</sub> MINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n<tr><td><i>max<sub>a in Action(s)</sub> MINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n</table>\n<br />\ne.g. a first <i>MIN</i> node has values 3, 12, and 8, so the minimax value is 3. The other successor states of the root have minimax values of 2,\nso the root has a minimax value of 3. The root is the <b>minimax decision</b> or optimal choice for <i>MAX</i> because it leads to the state with highest minimax value. The\nminimax algorithm works by computing the best move for <i>MAX</i> by choosing the state with the highest <i>MINIMAX</i> value. It recursively proceeds all the way down to\nthe leaves and then <b>backs up</b> the minimax values as the recursion unwinds. The time complexity is <i>O(b<sup>m</sup>)</i> and the space complexity is <i>O(bm)</i> when all\nactions are generated at once or <i>O(m)</i> when generated one at a time.\n<br /><br />\nIn multiplayer games, players form <b>alliances</b>, formally or informally, e.g. players A and B target C if C appears weaker.\n<br /><br />\nTo reduce the number of game states, <b>pruning</b> large parts of the tree that make no difference to the outcome can be done as in <b>alpha-beta pruning</b>. This involves\npassing additional parameters to <i>MAX_VALUE(state, alpha, beta)</i> where\n<ul><li><i>alpha</i> = the value of the best (i.e. highest-value) choice along the path for <i>MAX</i> (think: \"at least\")</li>\n<li><i>beta</i> = the value of the best (i.e. lwoest-value) choice along the path for <i>MIN</i> (think: \"at most\")</li></ul>\nThe branch factor can become <i>b<sup>0.5</sup></i> instead of <i>b</i>, so we only need to examine <i>O(b<sup>m/2</sup>)</i> instead of <i>O(b<sup>m</sup>)</i>\"\nnodes. Some other considerations include <b>iterative deepening</b> to expand a level before moving on to next, focusing on <b>killer moves</b> and try them first and to\nconsider using a <b>transposition table</b> that caches the heuristic value of states in the case of <b>transpositions</b> or permutations that result in the same game state.\n<b>Type A strategy</b> considers all possible moves to a certain depth and evaluates them using a heuristic while a <b>Type B Strategy</b> ignores moves that look bad,\ninstead focusing on a <i>deep but narrow</i> portion of the tree.\n\n## AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\nBecause computation time is limited, we cut off search early and use a heuristic <b>evaluation function</b> on the states, effectively treating them as terminal.\nWe replace the <i>UTILITY</i> function with the <i>EVAL</i> function, and also replace the terminal test with a <b>cutoff test</b> - which returns <i>True</i> for terminal\nstates or states past the cutoff depth. This gives the formula:\n<table>\n<tr><td><i>H-MINIMAX(s)</i>=</td><td></td></tr>\n<tr><td><i>EVAL(s,MAX)</i></td><td>if <i>IS-CUTOFF(s)</i></td></tr>\n<tr><td><i>max<sub>a in Action(s)</sub> H-MINIMAX(RESULT(s,a),d+1)</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n<tr><td><i>max<sub>a in Action(s)</sub> H-MINIMAX(RESULT(s,a),d+1)</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n</table>\n<br />\nFor terminal states, <i>EVAL(s,p)=UTILITY(s,p)</i> and for nonterminal states, <i>UTILITY(loss,p)<=EVAL(s,p)<=UTILITY(win,p)</i>. Most evaluation functions work by taking on \nthe <b>expected value</b> various <b>features</b> of the multiple <i>categories</i> or <i>equivalence classes</i>  and the <i>proportion</i> of each states with each outcome.\nA simple example is taking on the <b>material value</b> of each chess piece and then taking a <b>weighted linear function</b>, e.g. <i>EVAL(s) = w<sub>1</sub>f<sub>1</sub>(s) +\n... + w<sub>n</sub>f<sub>n</sub>(s)</i>. Some further modifications include applying the evaluation function only to <b>quiescent</b> positions where there is no pending move\nwhich swings the evaluation, and doing an extra <b>quiescence search</b> otherwise. There is also the <b>horizon effect</b> in which certain moves can be pushed off until\nlater, which can be mitigated by <b>singular extensions</b>, moves that are clearly better than others.\n<br /><br />\nAnother issue is with alpha-beta pruning potentially <b>forward pruning</b> some states that appear poor but are better. Some alternatives include <b>beam search</b> which\nonly includes the top <i>n</i> candidates but is similarly problematic, or methods like <i>PROBCUT</i> which considers moves <i>probably</i> instead of <i>provably</i> outside\nthe <i>(alpha, beta)</i> window. Another technique called <b>late move reduction</b> assumes moves are selected well, and discards moves that are later on. More complicated\nalgorithms like <i>STOCKFISH</i> also exist.\n<br /><br />\nFor openings and closings, a program can be trained to use a <i>table lookup</i> instead of calculating specific states. These include common chess openings. The table\ncan also be constructed by <b>retrogrdae</b> minimax search, which works backwards to identify particularly effective states.\n\n\n## AIMA Chapter 5.5 Stochastic Games\n<b>Stochastic games</b> introduce an element of chance by introducing random elements, e.g. throwing a dice or drawing a card. An example of this is Backgammon.\nBecause Black cannot know with certainty what outcome White will draw, the standard game tree from chess/tic-tac-toe cannot be used, but instead a game tree for Backgammon must\ninclude <b>chance nodes</b> in addition to the <i>MAX</i> and <i>MIN</i> nodes. The branches leading from each chance node represent the probability for each dice roll: while\nthere are 36 possible ways to roll two dice, each with 1/36 probability, order does not matter, i.e. a 4-2 is the same a s a 2-4. Therefore, there are only six distinct rolls.\nThe six doubles (e.g. 1-1,2-2,...6-6) each have probability of 1/36, e.g. P(3-3)=1/36, while the other 15 outcomes have a probability of 1/18.\n<br /><br />Now, since the goal is still to make the correct decision but the positions do not have definite values, we instead calculate the <b>expected value</b> of a\nposition, which is the average of all the possible outcomes over the chance nodes. This leads to the <b>expectiminimax value</b> for stochastic games, which is a generalization\nof the minimax value used with deterministic nodes. Terminal nodes for <i>MAX</i> and <i>MIN</i> work the same way as before. For chance nodes, the expected value which is the\nsum of the value over all outcomes, weighted by the likelihood or probability of each action:\n<table>\n<tr><td><i>EXPECTIMAX(s)</i>=</td><td></td></tr>\n<tr><td><i>UTILITY(s,MAX)</i></td><td>if <i>IS-TERMINAL(s)</i></td></tr>\n<tr><td><i>max(a) EXPECTIMINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n<tr><td><i>min(a) EXPECTIMINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n<tr><td><i>sum(r) P(r) * EXPECTIMINIMAX(RESULT(s,r))</i></td><td> if <i>TO-MOVE(s)=CHANCE</i></td></tr>\n</table>\nHere, <i>r</i> represents a possible dice roll (or other random event) and <i>RESULT(s, r)</i> is the state of <i>s</i> with the additional fact that the dice roll has\nresulted in <i>r</i>.<br /><br />Now, with approximate values, the evaluation functions do not just need to give higher values to better positions, but they need to be more\ncareful - the program behaves differently if a change is made to some of evaluation values even if preference remains the same, e.g. [2,3,4,5] vs [20,300,400,5000]. Hence,\ninstead, the evaluation function returns values that are a positive linear transformation of the expected utility - for games with outcomes of win/lose this is the \n<b>probability</b>  of winning.<br /><br />Now, if minimax can run in <i>O(b<sup>m</sup>)</i> time, where <i>b</i> is the branching factor and <i>m</i>is the maximum depth\nof the tree, then expectimax must always take into account the number <i>n</i> of distinct roles, resulting in a runtime of <i>O(b<sup>m</sup>n<sup>m</sup>)</i>. This makes it\nvery impractical to run more than a few ply of a game like Backgammon.\n\n## AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\nGiven a state <i>s</i>, we can say that the state has a probability associated with it that can be expressed as <i>P(s)</i>. Then for each action <i>a</i> that\nan agent can tak, the resulting state and the uncertainty of it can be expressed by <i>P(s'|s,a)</i>, i.e. the probability that the action <i>a</i> in state <i>s</i> leads to\nstate <i>s'</i>. Alternatively, because the focus is the agent reaching state <i>s'</i>, the notation <i>P(RESULT(a)=s')</i> can be used. This is related to all the current\nstates by the formula \n<br /><i>P(RESULT(a)=s') = sum<sub>s</sub>P(s)P(s'|s,a)</i><br />\nDecision theory concerns itself with finding the choice of action that has the best <i>immediate</i> result. In order to calculate this, the agent uses the <b>utility\nfunction</b> <i>U(s)</i> across all the potential states resulting from an action to calculate the <b>expected utility</b> <i>EU(a)</i>, which is also equivalent to\nthe average utility value of the outcomes weighted by the probability of each outcome occuring, i.e.\n<br /><i>EU(a) = sum<sub>s'</sub>P(RESULT(a)=s')U(s')</i><br />\nThe general principle that is followed is that of <b>maximum expected utility (MEU)</b> which states that a rational agent should seek to choose the action that provides\nmaximum utility, i.e.\n<br /><i>action = argmax<sub>a</sub>EU(a)</i><br />\nThis principle <i>formalizes</i> the general notion that a rational agent should do the optimal thing but does not <i>operationalzie</i> the action that does. Obtaining\na good utility function and perceiving the world is still a challenge and the agent must also be able to make the right actions based on the values it obtains from this.\nThis results in the somewhat tautological view that the agent that acts to maximize its utility function should correctly reflect its performance measure, resulting in\nthe agent obtaining the highest possible score when weighted against all of the potential environments. This correlates the external performance with its internal\nrepresentation.\n\n## AIMA Chapter 16.2 The Basis of Utility Theory\n\nWe can express an agent's preferences with the following notation:\n<table>\n<tr><td><i>A > B</i></td><td>the agent prefers <i>A</i> instead of <i>B</i></td></tr>\n<tr><td><i>A ~ B</i></td><td>the agent is indifferent between <i>A</i> and <i>B</i></td></tr>\n</table>\nFor a given action, can consider the set of possible outcomes each with respective probability of occuring a <b>lottery</b> <i>L</i> such that:\n<br /><i>L = [p<sub>1</sub>,S<sub>1</sub>;p<sub>2</sub>,S<sub>2</sub>;...;p<sub>n</sub>,S<sub>n</sub>]</i>\nEach outcome <i>S<sub>i</sub></i> can either be an atomic sentence or another lottery. There are six main axioms of utility theory:\n<ol><li><b>Orderability</b>: Exactly one of <i>A > B</i>, <i>B > A</i> or <i>A ~ B </i> holds.</li>\n<li><b>Transitivity</b>: <i>(A > B) /\\ (B > C) -> (A > C)</i></li>\n<li><b>Continuity</b>: If a lottery <i>B</i> is in between <i>A</i> and <i>C</i> in preference, then there is some probability <i>p</i> for which agent will be indifferent\nbetween getting <i>B</i> in a lottery that yields <i>A</i> with probability <i>p</i> and <i>C</i> with probability <i>1-p</i>:\n<br /><i>A > B > C -> exists(p) [p;A;1-p,C] ~ B</i></li>\n<li><b>Substitutability</b>: Given two lotteries, w.l.o.g. <i>A</i> and <i>B</i>, then the agent is indifferent between two or more complex\nlotteries such that the only difference is that <i>B</i> is substituted for <i>A</i> in one of them, i.e.\n<br /><i>A ~ b -> [p,A;1-p,C] ~ [p,B;1-p,C]</i></li>\n<li><b>Monotonicity</b>: Given two lotteries with the same two possible outcomes, w.l.o.g. <i>A</i> and <i>B</i>, if an agent prefers <i>A</i> to <i>B</i>,\nthe agent also prefers the lottery that has a higher probability of <i>A</i> (and vice versa):\n<br /><i>A > B -> (p > Q <-> [p,A;1-p,B] > [q,A;1-q,B])</i></li>\n<li><b>Decomposability</b>:Compound lotteries can be reduced to simpler ones using various laws of probability; for example, two consecutive lotteries can be condensed into\na single lottery as follows:<br /><i>[p,A;1-p,[q,B;1-q,C]] ~ [p,A;(1-p)q,B;(1-p)(1-q),C]</i></li>\n</ol>\nThese axioms of utility theory are:\n<ul>\n<li><b>Existence of utility Function</b>: if an agent has preferences that obey the axioms of utility, it follows that there exists a function <i>U</i> such that\n<i>(U(A) > U(B)) <-> (A > B)</i> and <i>(U(A) = U(B)) <-> (A ~ B)</i></li>\n<li><b>Expected Utility of a Lottery</b>: the sum of the probability of each outcome multipled by the utility of each outcome, i.e.\n<br /><i>U([p<sub>1</sub>,S<sub>1</sub>;...;p<sub>n</sub>,S<sub>n</sub>] = sum<sub>i</sub>p<sub>i</sub>U(S<sub>i</sub>)</i></li>\n</ul>\nThese theorems state that a utility function <i>exists</i> but not necessarily that an <i>unique</i> one exists. It is easy to see that any linear transformation, e.g.\n<i>U'(S) = aU(S) + b</i> is still valid. An agent only needs a preference ranking on these states not the actual values. This is called a <b>value function</b>\nor an <b>ordinal utility function</b>\n\n## AIMA Chapter 16.3 Utility Functions\nOne shortcoming of utility functions is they do not account for individual's preferences, which can lead to behavior that may initially appear irrational\nbut is substnatiated. These can be realized through a process called <b>preference elicitation</b>, whereby an agent is presented with different choices and then asked\nwhat their preferences are.\n<br /><br />\nIt is flawed to assume people have a <b>monotonic preference</b> for more money; given a gamble to wager 1 million for 2.5 by a coin flip, most people would not as the\n<b>expected money value</b> is not equal. For very rich or very poor people, they are more likely to gamble, indicating that the EMV of money is possibly logarithmic:\npeople at the curves are <b>risk-averse</b> while those at the fringes are <b>risk-seeking</b>.\n<br /><br />\nWe also do not perceive the world perfectly and are calculating the true utility. In reality, we are calculating an estimate of the utility function\n<i>EU<sub>est</sub>(a)</i> and in order for this to be <b>unbiased</b> the error must be 0, i.e. <i>E(EU<sub>est</sub>(a) - EU(a)) = 0</i>. We can often calculate the maximum\nof <i>k</i> estimates and then end up calculating an <b>order statistic</b> overestimate, for <i>X*</i> being the largest in a set {<i>X<sub>1</sub>,...X<sub>k</sub></i>},\n<br /><i>P</i>(max{<i>X<sub>1</sub>,...X<sub>k</sub></i>} <= <i>x</i>) = <i>F(x)<sup>k</sup></i>\n<br />and by its derivative,\n<br /><i>P(x) = kf(x)(F(x))<sup>k-1</sup></i>.\n<br /><br />\nThe <b>optimizer's curse</b> states that the estimated expected choice is likely to be too high. A study that shows a vaccine is 90% effective is not likely to be 90% effective on\nfuture victims for example, and most times something that is effective on 9/10 patients in fact performs worse than something else which is effective on 8000/10000\npatients. We can aim to mitigate this with a Bayesian approach that uses an explicit probability model <b>P</b>(<i>EU<sub>est</sub>|EU</i>) of the error in utility rates.\n<br /><br />\nSome phenomenon that can be observed include:\n<ul><li>the <b>certainty effect</b>: people are more likely to choose certain outcomes over ambiguous outcomes that may have a higher utility value (even in expectation).\nPeople can be described as having <b>ambiguity aversion</b></li>\n<li>the <b>framing effect</b>: people are more likely to respond positively to a large positive number than a small negative number, e.g. a 80% chance of living is seen\nas better than a 20% chance of dying even though those are the same.</li>\n<li>the <b>anchoring effect</b>: people judge based on relative not absolute utility. Pricing an expensive item like a 300 dollar bottle of wine or an overpriced medium box of\npopcorn drives people to get the 55 dollar bottle or the large popcorn.</li>\nInvestigation into phenomena like this from the perspective of <b>evolutionary psychology</b> indicates that the brain's inherent decision making systems may not view the\nworld through probabilities.\n\n## Module 5 Lecture Notes\n Your summary here \n\n## Module 5 Definitions\n* **Transition model** - the resulting state after an action is taken\n* **Utility function** - function for the numeric value assigned to a player when the game terminates\n* **Expected utility** - the average value of the utility function over all the potential outcomes\n* **Maximum expected utility** - the principle that a rational agent should choose actions that maximize expected utility\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qVSQhTVSyYYO"
      },
      "source": [
        "#Module 6 - \"Constraint Satisfaction Problems\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wm4mfW82yaRH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5fd6f61e-f4bc-472e-a55b-bea0504e9bbc"
      },
      "source": [
        "# Russell and Norvig, AIMA, Chapter 6 “Constraint Satisfaction Problems” (6.1-6.5)\n",
        "chapter6_1 = \"\"\" Solution to a CSP must be both <b>consistent</b> and <b>complete</b>.\n",
        "<br /><br />\n",
        "In the Map Coloring problem, there are constraints that different territories cannot be the same color as their adjacent territories. We can come up wtih constraints in the\n",
        "form <i>C = {(SA, WA), SA =/= WA}</i> and from this reduce the map to each territory having its own color. One helpful tool is a <b>constraint graph</b> where nodes are the\n",
        "variables and edges are possible solutions. Reducing constraints can significantly reduce the output space 87%.\n",
        "<br /><br />\n",
        "In the Job Scheduling problem, certain operations must occur before others. These operations may take a specific amount of time. These result in <b>precedence constraints</b> \n",
        "that say a task must occur a certain amount of time before another, e.g. <i>T<sub>1</sub></i> + <i>d<sub>1</sub></i> <= <i>T<sub>2</sub></i> indicates task <i>T<sub>1</sub></i> \n",
        "takes <i>d<sub>1</sub></i> time to complete and must come before <i>T<sub>2</sub></i>. Another type of constraint are the <b>disjunctive constraint</b> which specify one or \n",
        "another action must come before the other, e.g. <i>T<sub>1</sub></i> + <i>d<sub>1</sub></i> <= <i>T<sub>2</sub></i> <b>or</b> <i>T<sub>3</sub></i> + <i>d<sub>3</sub></i> <= \n",
        "<i>T<sub>4</sub></i>.\n",
        "<br /><br />\n",
        "There are several types of constraints:\n",
        "<ul><li><b>unary constraint</b>: A constraint that restricts the value of a single variable.</li>\n",
        "<li><b>binary constraint</b>: A constraint that relates two variables. A problem that is made up solely of unary and binary constraints is a <b>binary CSP</b>.</li>\n",
        "<li><b>global constraint</b>: A constraint which applies across an arbitrary number of variables. A common global constraint is <i>Alldif</i> which specifies each element\n",
        "must be different, e.g. the elements in a sudoku box. Another example are <b>cryptarithmetic</b> puzzles. These can often be modeled as a <b>constraint hypergraph</b>\n",
        "where special nodes indicate <i>n</i>-ary relationships. Alternatively, auxiliary variables can be introduced to reduce these to binary constraints and a <b>dual graph</b>\n",
        "can be used instead.</li></ul>\n",
        "The above examples involve <b>discrete, finite domains</b>. Other problems that can come up include <b>infinite domains</b> which require relationships to be expressed in the\n",
        "form of <b>linear constraints</b> on integer variables (as opposed to explicitly defined tuples). It has been shown the solution when including <b>nonlinear constraints</b> is\n",
        "undecidable. Another variation involves using <b>continuous domains</b>, e.g. scheduling problems with very specific time constraints. These form the basis of <b>linear\n",
        "programming</b> problems. A third variation involves specifying <b>preference constraints</b> which indicate which solutions might be preferred. These can be solved by using\n",
        "optimization search methods that can be either path-based or local and are called a <b>constraint optimization problem</b>. Linear programs are a type of COP.\n",
        "\"\"\"\n",
        "chapter6_2 = \"\"\"<b>Constraint satisfaction</b> aims to ensure <b>local consistency</b> including:\n",
        "<ul>\n",
        "<li><b>Node consistency</b> ensures a node meets its unary constraints.</li>\n",
        "<li><b>Arc consistency</b> holds if every value in the domain satisfies its binary constraints. AC-3 enforces this by: \n",
        "<ul><li>Initially, all the arcs in a CSP are placed in a queue.</li>\n",
        "<li>AC-3 then pops off a queue and makes its two nodes arc-consistent.</li>\n",
        "<li>If any node changes, enque all its neighboring arcs.</li>\n",
        "<li>If no values for a node, AC-3 terminates.</li></ul>\n",
        "Runtime is <i>O(cd<sup>3</sup>)</i>.</li>\n",
        "<li><b>Path consistency</b> uses implicit constraints that can be inferred by looking at triples of variables on binary constraints, i.e. ensuring that for a two-variable\n",
        "assignment {<i>X<sub>i</sub> = a, X<sub>j</sub> = b</i>}, an assignment is consistent iff there is an assignment to <i>X<sub>m</sub></i> that satisfies both \n",
        "{<i>X<sub>i</sub>, X<sub>m</sub></i>} and {<i>X<sub>m</sub>, X<sub>j</sub></i>}.\n",
        "<li><b><i>k</i>-consistency</b> <-> for any consistent assignment to a set of <i>k - 1</i> variables, the <i>k</i>-th value can always be assigned a consistent value.\n",
        "This is a stronger form of propagation: 1-consistency is node consistency, 2-consistency is arc concistency, in a binary graph 3-consistency is path-consistency. A CSP\n",
        "is further <b>strongly <i>k</i>-consistent</b> if it is <i>k</i>-consistent, (<i>k</i>-1)-consistent, ..., 1-consistent. This means that for a <i>n</i>> node problem,\n",
        "we choose a consistent value for <i>X<sub>1</sub></i> and are guaranteed consistent solutions for <i>X<sub>2</sub>,...,X<sub>n</sub></i> by 2-consistency, 3-consistency, etc.\n",
        "Therefore the time is <i>O(n<sup>2</sup>d)</i> but establishing <i>n</i>-consistency is NP-complete.</li>\n",
        "<li><b>global constraint</b></li>\n",
        "</ul>\n",
        "Another high-order constraint is the <b>resource constraint</b> or the <i>Atmost</i> constraint, which specifies a maximum total for a subset of nodes, e.g. for 4 people\n",
        " if we have that <i>Atmost(10,P<sub>1</sub>,P<sub>2</sub>,P<sub>3</sub>,P<sub>4</sub>)</i> then we know for a sample domain {2,3,5,6}\n",
        "will not satisfy this since 5 + 6 > 10, so we can remove 5 and 6 from all domains involving those 4.\n",
        "<br /><br />\n",
        "For large domains that cannot be easily enumerated, we use <b>bounds propagation</b> by keeping track of the range that a variable can be in, e.g.\n",
        "<i>D<sub>1</sub> = [0,165]</i> and adjust this value as new constraints appear. A CSP is <b.bounds-consistent</b> if for every <i>X</i> variable in the CSP, there is a variable\n",
        "<i>Y</i> that satisfies the constraint between <i>X</i> and <i>Y</i>.\n",
        "<br /><br />\n",
        "<b>Sudoku</b> is a puzzle that involves 81 tiles and 27 <i>Alldiff</i> <b>units</b>: one for each row, column, and 3x3 box. AC-3 works for the easiest of Sudoku puzzles, PC-2 \n",
        "can solve harder puzzles but with 255,960+ path constraints, more complicated strategies like isolating \"naked triples\" and removing them from other arcs required.\n",
        "\"\"\"\n",
        "chapter6_3 = \"\"\"If we finish CSP process with multiple values, then need to <b>search</b> for a solution. One possible way is to do depth-limited search. But for domain\n",
        "of size <i>d</i> with leaf nodes at depth <i>n</i>, branch factor would be <i>nd</i> because <i>d</i> values can be assigned to any of <i>n</i> levels, so there are\n",
        "<i>n!*d<sup>n</sup></i> leaves for <i>d<sup>n</sup></i> arrangements. However, CSPs are <b>commutative</b> and so backtracking by a series of heuristics can get back the\n",
        "<i>n!</i> factor.\n",
        "<br /><br />\n",
        "Some important heuristics to consider are the <b>minimmum-remining-values</b> (MRV) heuristic, aka the \"most constrained variable\" or \"fail first\" heuristic. This is the\n",
        "variable that is most likely to fail, as it has the fewest options, so the search can terminate earlier if it is deteced immediately (if it has no legal values). This\n",
        "prunes the search tree by minimizing the number of nodes expanded. Another heuristic is the <b>degree heuristic</b> by selecting the variables that have constraints on the\n",
        "largest number of other unassigned variables. This is not as good as MRV but can be used as a deal-breaker.\n",
        "<br /><br />\n",
        "After a variable has been selected, the algorithm decides which order to examine the values by using the <b>least-constraining-value</b> heuristic, or picking the value that\n",
        "rules out the fewest choices for neighbors in the constraint graph, or fail-last. The dichotomy between the two is fail-first minimizes the number of nodes to expand but\n",
        "fail-last then maximizes the number of possible outcomes that can be carried out forward.\n",
        "<br /><br />\n",
        "Running inference during the process of search can yield new domain restrictions. One is <b>forward checking</b>, whereby after a value is assigned,\n",
        "we check the arc consistency for it. This is often more effective when combined with the MRV heuristic, making it an efficient way to incrementally compute the information\n",
        "that will be used by MRV. It does not detect all the possibilities however, so stronger algorithms like <b>Maintaining Arc Consistency</b> (MAC) are useful this. It calls\n",
        "AC-3 again but without the entire queue just the values that are unassigned neighbors of <i>X<sub>i</sub></i>, then doing constraint propagation. If any variable has its domain\n",
        "reduced to empty, this backtracks immediately. This recursive propogation makes it more powerful than just forward checking alone.\n",
        "<br /><br />\n",
        "When a branch fails, a process of <b>chronological backtracking</b> to the <i>most recent</i> decision is done. Some better psosibilities are to instead look at the\n",
        "<b>conflict set</b> that causes the CSP to fail, and then <b>backjump</b> until an assignment in that set fails, and then try out the other possible values for this assignment.\n",
        "If none are found, then the CSP will fail. However, every branch trimmed by backjumping would also be checked by forward pruning, so only one is needed.\n",
        "<br /><br />\n",
        "<b>Constraint learning</b> can further  avoid <b>no-good</b> assignments.\n",
        "\"\"\"\n",
        "chapter6_4 = \"\"\"Local search algorithms perform significantly better with CSPs. In one example, for the 8-queens problem, instead of starting with a valid placement for the\n",
        "leftmost assignment, which will typically create several assignments, we can instead choose a randomly conflicted variable, and change this to a value that will bring it\n",
        "closer to the solution using the <b>min-conflicts</b> heuristic, which selects the value that results in the minimum conflicts with other variables. This is surprisingly\n",
        "effective at many CSPs, excluding the initial placement of queens, its runtime is more or less <i>independent of problem size</i> - it can solve the <i>million</i>-queens\n",
        "problem in about 50 steps after the initial assignment. Research has shown this is because the solutions are densely distributed through the state space. Still, local\n",
        "search has shown potential for planning the Hubble Space Telescope observations and harder problems.\n",
        "<br /><br />\n",
        "One challenge is that there could be millions of variable assignments that are only one conflict away from a solution. Plateau search, or methods that would allow one to\n",
        "move to another state with the same exact score could help a local search off its plateau. This can be directed via methods like <b>tabu search</b>, which keeps a small\n",
        "list of recently visited states and forbids revisiting those states. To escape from plateaus, simulated annealing can also be used.\n",
        "<br /><br />\n",
        "A final technique called <b>constraint weighting</b> places more weight on the most important constraints. All of the constraints are initially given a weight of 1, but\n",
        "after each step in the local search algorithm, the algorithm will choose a variable and value pair to change and that will as a result have the lowest total weight of all\n",
        "the violated constraints considered so far. Weights would then be adjusted by incrementing each constraint violated by the current assignment. This has two benefits:\n",
        "first, it adds topography to plateaus, making it possible to improve from the current state, and it adds learning: over time the higher priority constraints get assigned\n",
        "higher values.\"\"\"\n",
        "chapter6_5 = \"\"\"Finding <b>independent subproblems</b> can have a significant impact on being able to find the optimal solution. Breaking a graph up into <b>connnected\n",
        "components</b> significantly reduces the runtime - given <i>c</i> out of <i>n</i> variables where there are <i>n/c</i> subproblems and <i>d</i> amount of work to solve \n",
        "overall total, or <i>d<sup>c</sup></i> per component, the total work is <i>O(d<sup>c</sup>n/c)</i>, which is <i>linear</i> in\n",
        "<i>n</i>. Without the decomposition it is <i>O(d<sup>c</sup>)</i>, which is <i>exponential</i> in <i>n</i>.\n",
        "<br /><br />\n",
        "While completely independent subproblems are rare, there are other structures that are helpful, e.g. a <b>tree</b> can be solved in time linear\n",
        "in the number of variables. This is because of <b>directional arc consistency</b> or DAC. A CSP is consistent for variables if and only if every \n",
        "<i>X<sub>i</sub></i> is arc consistent with <i>X<sub>j</sub></i> for <i>j > i</i>. In order to do this, one picks a variable to be a root\n",
        "of tree and then orders the tree such that each variable appears after its parent in a tree, i.e. a <b>topological sort</b>. A tree with <i>n</i> nodes has <i>n - 1</i> edges\n",
        "and thus the graph can be made arc-consistent in <i>O(n)</i> steps, each of which will consider <i>d</i> possible domains for a runtime of <i>O(nd<sup>2</sup>)</i>.\n",
        "<br /><br />\n",
        "<b>Cutset conditioning</b> is a technique of assigning values to some variables so the remainder form a tree. If a cycle cutset has size <i>c</i>, then the total\n",
        "runtime of the following is <i>O(d<sup>c</sup>*(n-c)d<sup>2</sup>)</i>:\n",
        "<ol><li>Choose a subset <i>S</i> of the CSP's variables such that the residual constraint graph is a tree. then<i>S</i> is called a <b>cycle cutset</b>.</li>\n",
        "<li>For each possible assignment to the variables in <i>S</i> which satisfy all the constraints on <i>S</i>,\n",
        "<ol><li>remove from the domains of the remaining variables any values that are inconsistent with the assignemnt for <i>S</i>, and</li>\n",
        "<li>if the remaining CSP has a solution, return it together with the assignment for <i>S</i>.</li></ol></li></ol>\n",
        "A <b>tree decomposition</b> of the constraint graph is a transformation of it into a tree where each node contains a set of variables. It must follow three requirements:\n",
        "<ol><li>Every variable in the problem appears in at least one of the tree nodes.</li>\n",
        "<li>If two variables are connected by a constraint in the original problem, they must appear together (along with the constraint) in at least one of the tree nodes.</li>\n",
        "<li>If a variable appears in two nodes in the tree, it must appear in every node along the path connecting those nodes.</li></ol>\n",
        "Tree decomposition has a runtime of <i>O(nd<sup>w+1</sup>)</i>, so whenever the cycle subset <i>c</i> and tree width <i>w</i> are of size <i>w < c + 1</i>, tree\n",
        "decomposition is better.\n",
        "<br /><br />\n",
        "A final method is <b>value symmetry</b> which breaks the symmetry in assignments using a <b>symmetry-breaking constraint</b>.\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "# Lecture notes \n",
        "module6_lecture_notes = \"\"\" Your summary here \"\"\"\n",
        "\n",
        "print(\"The length in words is\", len(chapter6_5.split(' ')))\n",
        "print(\"The length in characters is\", len(chapter6_5))\n",
        "\n",
        "# Definitions\n",
        "module6_definitions = {\n",
        "\"factored representation\" : \"a representation of a problem via a set of variables, each with a value\",\n",
        "\"constraint satisfaction problem (CSP)\" : \"a pproblem that is solved when each of its variables has a value and the original constraints are satisfied\",\n",
        "\"relation\" : \"a tuple relationship between a variable and the set of values it can take on\",\n",
        "\"assignment\" : \"a set of values given to a set of variables\",\n",
        "\"consistent\" : \"an assignment that does not violate any of the constraints\",\n",
        "\"complete assignment\" : \"an assignment in which each variable has a value\",\n",
        "\"partial assignment\" : \"an assignment that leaves some variables incomplete\",\n",
        "\"partial solution\" : \"a partial assignment that is consistent\",\n",
        "\"constraint graph\" : \"a visualization of a CSP where nodes correspond to variables and edges represent constraints\",\n",
        "\"precedence constraint\" : \"a constraint on when an individual task can occur or how long it can take\",\n",
        "\"disjunctive constraint\" : \"a constraint that allows for one of multiple possibilties (logical OR)\",\n",
        "\"discrete domain\" : \"a domain involving real numbers, e.g. map-coloring and scheduling with time limits\",\n",
        "\"continuous domain\" : \"a domain involving non-exact start and end times, like specialized scheduling tasks\",\n",
        "\"finite domain\" : \"a domain that has an upper limit to its size\",\n",
        "\"infinite domain\" : \"a domain that has no known upper limit, e.g. the set of integers or strings\",\n",
        "\"linear constraint\" : \"a constraint in which each variable is expressed in linear from with a linear relationship\",\n",
        "\"linear programming\" : \"a type of continuous-domain CSP where constraints must be linear equalities or inequalities\",\n",
        "\"unary constraint\" : \"a constraint on a single value or variable\",\n",
        "\"binary constraint\" : \"a constraint that relates two variables\",\n",
        "\"binary CSP\" : \"a CSP involving only unary and binary constraints\",\n",
        "\"global constraint\" : \"a constraint involving an arbitrary number of variables\",\n",
        "\"cryptarithmetic\" : \"a CSP involving assigning values to letters based on linear equations\",\n",
        "\"constraint hypergraph\" : \"a constraight graph for higher-order csps, where n-ary constraints are represented as nodes\",\n",
        "\"dual graph\" : \"a graph in which one variable is in one graph and the constraints are in another graph\",\n",
        "\"preference constraint\" : \"a constraint that expresses a preference but not an absolute trestriction\",\n",
        "\"constrained optimization problem\" : \"csps involving preferences that can be solved using optimization search methods\"\n",
        "}\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 429\n",
            "The length in characters is 2980\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "kKWbCsbfxL3R",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 2858
        },
        "outputId": "1ae9a818-a939-40f2-8133-dae6cc12bf19"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module6_sections = [\n",
        "  (\"AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\", chapter6_1),\n",
        "  (\"AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\", chapter6_2),\n",
        "  (\"AIMA Chapter 6.3 Backtracking Search for CSPs\", chapter6_3),\n",
        "  (\"AIMA Chapter 6.4 Local Search for CSPs\", chapter6_4),\n",
        "  (\"AIMA Chapter 6.5 The Structure of Problems\", chapter6_5),\n",
        "  (\"Module 6 Lecture Notes\", module6_lecture_notes),\n",
        "]\n",
        "\n",
        "module6_markdown = format_module(6, \"Constraint Satisfaction Problems\", \n",
        "                         module6_sections, \n",
        "                         module6_definitions)\n",
        "display(Markdown(module6_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 6: Constraint Satisfaction Problems\n## AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\n Solution to a CSP must be both <b>consistent</b> and <b>complete</b>.\n<br /><br />\nIn the Map Coloring problem, there are constraints that different territories cannot be the same color as their adjacent territories. We can come up wtih constraints in the\nform <i>C = {(SA, WA), SA =/= WA}</i> and from this reduce the map to each territory having its own color. One helpful tool is a <b>constraint graph</b> where nodes are the\nvariables and edges are possible solutions. Reducing constraints can significantly reduce the output space 87%.\n<br /><br />\nIn the Job Scheduling problem, certain operations must occur before others. These operations may take a specific amount of time. These result in <b>precedence constraints</b> \nthat say a task must occur a certain amount of time before another, e.g. <i>T<sub>1</sub></i> + <i>d<sub>1</sub></i> <= <i>T<sub>2</sub></i> indicates task <i>T<sub>1</sub></i> \ntakes <i>d<sub>1</sub></i> time to complete and must come before <i>T<sub>2</sub></i>. Another type of constraint are the <b>disjunctive constraint</b> which specify one or \nanother action must come before the other, e.g. <i>T<sub>1</sub></i> + <i>d<sub>1</sub></i> <= <i>T<sub>2</sub></i> <b>or</b> <i>T<sub>3</sub></i> + <i>d<sub>3</sub></i> <= \n<i>T<sub>4</sub></i>.\n<br /><br />\nThere are several types of constraints:\n<ul><li><b>unary constraint</b>: A constraint that restricts the value of a single variable.</li>\n<li><b>binary constraint</b>: A constraint that relates two variables. A problem that is made up solely of unary and binary constraints is a <b>binary CSP</b>.</li>\n<li><b>global constraint</b>: A constraint which applies across an arbitrary number of variables. A common global constraint is <i>Alldif</i> which specifies each element\nmust be different, e.g. the elements in a sudoku box. Another example are <b>cryptarithmetic</b> puzzles. These can often be modeled as a <b>constraint hypergraph</b>\nwhere special nodes indicate <i>n</i>-ary relationships. Alternatively, auxiliary variables can be introduced to reduce these to binary constraints and a <b>dual graph</b>\ncan be used instead.</li></ul>\nThe above examples involve <b>discrete, finite domains</b>. Other problems that can come up include <b>infinite domains</b> which require relationships to be expressed in the\nform of <b>linear constraints</b> on integer variables (as opposed to explicitly defined tuples). It has been shown the solution when including <b>nonlinear constraints</b> is\nundecidable. Another variation involves using <b>continuous domains</b>, e.g. scheduling problems with very specific time constraints. These form the basis of <b>linear\nprogramming</b> problems. A third variation involves specifying <b>preference constraints</b> which indicate which solutions might be preferred. These can be solved by using\noptimization search methods that can be either path-based or local and are called a <b>constraint optimization problem</b>. Linear programs are a type of COP.\n\n\n## AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\n<b>Constraint satisfaction</b> aims to ensure <b>local consistency</b> including:\n<ul>\n<li><b>Node consistency</b> ensures a node meets its unary constraints.</li>\n<li><b>Arc consistency</b> holds if every value in the domain satisfies its binary constraints. AC-3 enforces this by: \n<ul><li>Initially, all the arcs in a CSP are placed in a queue.</li>\n<li>AC-3 then pops off a queue and makes its two nodes arc-consistent.</li>\n<li>If any node changes, enque all its neighboring arcs.</li>\n<li>If no values for a node, AC-3 terminates.</li></ul>\nRuntime is <i>O(cd<sup>3</sup>)</i>.</li>\n<li><b>Path consistency</b> uses implicit constraints that can be inferred by looking at triples of variables on binary constraints, i.e. ensuring that for a two-variable\nassignment {<i>X<sub>i</sub> = a, X<sub>j</sub> = b</i>}, an assignment is consistent iff there is an assignment to <i>X<sub>m</sub></i> that satisfies both \n{<i>X<sub>i</sub>, X<sub>m</sub></i>} and {<i>X<sub>m</sub>, X<sub>j</sub></i>}.\n<li><b><i>k</i>-consistency</b> <-> for any consistent assignment to a set of <i>k - 1</i> variables, the <i>k</i>-th value can always be assigned a consistent value.\nThis is a stronger form of propagation: 1-consistency is node consistency, 2-consistency is arc concistency, in a binary graph 3-consistency is path-consistency. A CSP\nis further <b>strongly <i>k</i>-consistent</b> if it is <i>k</i>-consistent, (<i>k</i>-1)-consistent, ..., 1-consistent. This means that for a <i>n</i>> node problem,\nwe choose a consistent value for <i>X<sub>1</sub></i> and are guaranteed consistent solutions for <i>X<sub>2</sub>,...,X<sub>n</sub></i> by 2-consistency, 3-consistency, etc.\nTherefore the time is <i>O(n<sup>2</sup>d)</i> but establishing <i>n</i>-consistency is NP-complete.</li>\n<li><b>global constraint</b></li>\n</ul>\nAnother high-order constraint is the <b>resource constraint</b> or the <i>Atmost</i> constraint, which specifies a maximum total for a subset of nodes, e.g. for 4 people\n if we have that <i>Atmost(10,P<sub>1</sub>,P<sub>2</sub>,P<sub>3</sub>,P<sub>4</sub>)</i> then we know for a sample domain {2,3,5,6}\nwill not satisfy this since 5 + 6 > 10, so we can remove 5 and 6 from all domains involving those 4.\n<br /><br />\nFor large domains that cannot be easily enumerated, we use <b>bounds propagation</b> by keeping track of the range that a variable can be in, e.g.\n<i>D<sub>1</sub> = [0,165]</i> and adjust this value as new constraints appear. A CSP is <b.bounds-consistent</b> if for every <i>X</i> variable in the CSP, there is a variable\n<i>Y</i> that satisfies the constraint between <i>X</i> and <i>Y</i>.\n<br /><br />\n<b>Sudoku</b> is a puzzle that involves 81 tiles and 27 <i>Alldiff</i> <b>units</b>: one for each row, column, and 3x3 box. AC-3 works for the easiest of Sudoku puzzles, PC-2 \ncan solve harder puzzles but with 255,960+ path constraints, more complicated strategies like isolating \"naked triples\" and removing them from other arcs required.\n\n\n## AIMA Chapter 6.3 Backtracking Search for CSPs\nIf we finish CSP process with multiple values, then need to <b>search</b> for a solution. One possible way is to do depth-limited search. But for domain\nof size <i>d</i> with leaf nodes at depth <i>n</i>, branch factor would be <i>nd</i> because <i>d</i> values can be assigned to any of <i>n</i> levels, so there are\n<i>n!*d<sup>n</sup></i> leaves for <i>d<sup>n</sup></i> arrangements. However, CSPs are <b>commutative</b> and so backtracking by a series of heuristics can get back the\n<i>n!</i> factor.\n<br /><br />\nSome important heuristics to consider are the <b>minimmum-remining-values</b> (MRV) heuristic, aka the \"most constrained variable\" or \"fail first\" heuristic. This is the\nvariable that is most likely to fail, as it has the fewest options, so the search can terminate earlier if it is deteced immediately (if it has no legal values). This\nprunes the search tree by minimizing the number of nodes expanded. Another heuristic is the <b>degree heuristic</b> by selecting the variables that have constraints on the\nlargest number of other unassigned variables. This is not as good as MRV but can be used as a deal-breaker.\n<br /><br />\nAfter a variable has been selected, the algorithm decides which order to examine the values by using the <b>least-constraining-value</b> heuristic, or picking the value that\nrules out the fewest choices for neighbors in the constraint graph, or fail-last. The dichotomy between the two is fail-first minimizes the number of nodes to expand but\nfail-last then maximizes the number of possible outcomes that can be carried out forward.\n<br /><br />\nRunning inference during the process of search can yield new domain restrictions. One is <b>forward checking</b>, whereby after a value is assigned,\nwe check the arc consistency for it. This is often more effective when combined with the MRV heuristic, making it an efficient way to incrementally compute the information\nthat will be used by MRV. It does not detect all the possibilities however, so stronger algorithms like <b>Maintaining Arc Consistency</b> (MAC) are useful this. It calls\nAC-3 again but without the entire queue just the values that are unassigned neighbors of <i>X<sub>i</sub></i>, then doing constraint propagation. If any variable has its domain\nreduced to empty, this backtracks immediately. This recursive propogation makes it more powerful than just forward checking alone.\n<br /><br />\nWhen a branch fails, a process of <b>chronological backtracking</b> to the <i>most recent</i> decision is done. Some better psosibilities are to instead look at the\n<b>conflict set</b> that causes the CSP to fail, and then <b>backjump</b> until an assignment in that set fails, and then try out the other possible values for this assignment.\nIf none are found, then the CSP will fail. However, every branch trimmed by backjumping would also be checked by forward pruning, so only one is needed.\n<br /><br />\n<b>Constraint learning</b> can further  avoid <b>no-good</b> assignments.\n\n\n## AIMA Chapter 6.4 Local Search for CSPs\nLocal search algorithms perform significantly better with CSPs. In one example, for the 8-queens problem, instead of starting with a valid placement for the\nleftmost assignment, which will typically create several assignments, we can instead choose a randomly conflicted variable, and change this to a value that will bring it\ncloser to the solution using the <b>min-conflicts</b> heuristic, which selects the value that results in the minimum conflicts with other variables. This is surprisingly\neffective at many CSPs, excluding the initial placement of queens, its runtime is more or less <i>independent of problem size</i> - it can solve the <i>million</i>-queens\nproblem in about 50 steps after the initial assignment. Research has shown this is because the solutions are densely distributed through the state space. Still, local\nsearch has shown potential for planning the Hubble Space Telescope observations and harder problems.\n<br /><br />\nOne challenge is that there could be millions of variable assignments that are only one conflict away from a solution. Plateau search, or methods that would allow one to\nmove to another state with the same exact score could help a local search off its plateau. This can be directed via methods like <b>tabu search</b>, which keeps a small\nlist of recently visited states and forbids revisiting those states. To escape from plateaus, simulated annealing can also be used.\n<br /><br />\nA final technique called <b>constraint weighting</b> places more weight on the most important constraints. All of the constraints are initially given a weight of 1, but\nafter each step in the local search algorithm, the algorithm will choose a variable and value pair to change and that will as a result have the lowest total weight of all\nthe violated constraints considered so far. Weights would then be adjusted by incrementing each constraint violated by the current assignment. This has two benefits:\nfirst, it adds topography to plateaus, making it possible to improve from the current state, and it adds learning: over time the higher priority constraints get assigned\nhigher values.\n\n## AIMA Chapter 6.5 The Structure of Problems\nFinding <b>independent subproblems</b> can have a significant impact on being able to find the optimal solution. Breaking a graph up into <b>connnected\ncomponents</b> significantly reduces the runtime - given <i>c</i> out of <i>n</i> variables where there are <i>n/c</i> subproblems and <i>d</i> amount of work to solve \noverall total, or <i>d<sup>c</sup></i> per component, the total work is <i>O(d<sup>c</sup>n/c)</i>, which is <i>linear</i> in\n<i>n</i>. Without the decomposition it is <i>O(d<sup>c</sup>)</i>, which is <i>exponential</i> in <i>n</i>.\n<br /><br />\nWhile completely independent subproblems are rare, there are other structures that are helpful, e.g. a <b>tree</b> can be solved in time linear\nin the number of variables. This is because of <b>directional arc consistency</b> or DAC. A CSP is consistent for variables if and only if every \n<i>X<sub>i</sub></i> is arc consistent with <i>X<sub>j</sub></i> for <i>j > i</i>. In order to do this, one picks a variable to be a root\nof tree and then orders the tree such that each variable appears after its parent in a tree, i.e. a <b>topological sort</b>. A tree with <i>n</i> nodes has <i>n - 1</i> edges\nand thus the graph can be made arc-consistent in <i>O(n)</i> steps, each of which will consider <i>d</i> possible domains for a runtime of <i>O(nd<sup>2</sup>)</i>.\n<br /><br />\n<b>Cutset conditioning</b> is a technique of assigning values to some variables so the remainder form a tree. If a cycle cutset has size <i>c</i>, then the total\nruntime of the following is <i>O(d<sup>c</sup>*(n-c)d<sup>2</sup>)</i>:\n<ol><li>Choose a subset <i>S</i> of the CSP's variables such that the residual constraint graph is a tree. then<i>S</i> is called a <b>cycle cutset</b>.</li>\n<li>For each possible assignment to the variables in <i>S</i> which satisfy all the constraints on <i>S</i>,\n<ol><li>remove from the domains of the remaining variables any values that are inconsistent with the assignemnt for <i>S</i>, and</li>\n<li>if the remaining CSP has a solution, return it together with the assignment for <i>S</i>.</li></ol></li></ol>\nA <b>tree decomposition</b> of the constraint graph is a transformation of it into a tree where each node contains a set of variables. It must follow three requirements:\n<ol><li>Every variable in the problem appears in at least one of the tree nodes.</li>\n<li>If two variables are connected by a constraint in the original problem, they must appear together (along with the constraint) in at least one of the tree nodes.</li>\n<li>If a variable appears in two nodes in the tree, it must appear in every node along the path connecting those nodes.</li></ol>\nTree decomposition has a runtime of <i>O(nd<sup>w+1</sup>)</i>, so whenever the cycle subset <i>c</i> and tree width <i>w</i> are of size <i>w < c + 1</i>, tree\ndecomposition is better.\n<br /><br />\nA final method is <b>value symmetry</b> which breaks the symmetry in assignments using a <b>symmetry-breaking constraint</b>.\n\n\n## Module 6 Lecture Notes\n Your summary here \n\n## Module 6 Definitions\n* **Factored representation** - a representation of a problem via a set of variables, each with a value\n* **Constraint satisfaction problem (csp)** - a pproblem that is solved when each of its variables has a value and the original constraints are satisfied\n* **Relation** - a tuple relationship between a variable and the set of values it can take on\n* **Assignment** - a set of values given to a set of variables\n* **Consistent** - an assignment that does not violate any of the constraints\n* **Complete assignment** - an assignment in which each variable has a value\n* **Partial assignment** - an assignment that leaves some variables incomplete\n* **Partial solution** - a partial assignment that is consistent\n* **Constraint graph** - a visualization of a CSP where nodes correspond to variables and edges represent constraints\n* **Precedence constraint** - a constraint on when an individual task can occur or how long it can take\n* **Disjunctive constraint** - a constraint that allows for one of multiple possibilties (logical OR)\n* **Discrete domain** - a domain involving real numbers, e.g. map-coloring and scheduling with time limits\n* **Continuous domain** - a domain involving non-exact start and end times, like specialized scheduling tasks\n* **Finite domain** - a domain that has an upper limit to its size\n* **Infinite domain** - a domain that has no known upper limit, e.g. the set of integers or strings\n* **Linear constraint** - a constraint in which each variable is expressed in linear from with a linear relationship\n* **Linear programming** - a type of continuous-domain CSP where constraints must be linear equalities or inequalities\n* **Unary constraint** - a constraint on a single value or variable\n* **Binary constraint** - a constraint that relates two variables\n* **Binary csp** - a CSP involving only unary and binary constraints\n* **Global constraint** - a constraint involving an arbitrary number of variables\n* **Cryptarithmetic** - a CSP involving assigning values to letters based on linear equations\n* **Constraint hypergraph** - a constraight graph for higher-order csps, where n-ary constraints are represented as nodes\n* **Dual graph** - a graph in which one variable is in one graph and the constraints are in another graph\n* **Preference constraint** - a constraint that expresses a preference but not an absolute trestriction\n* **Constrained optimization problem** - csps involving preferences that can be solved using optimization search methods\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RYESi2kMybL8"
      },
      "source": [
        "#Module 7 - Logical Agents"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qIRJ4YYCzNHL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5a974459-1728-4d01-dba1-ac3f4b4f7097"
      },
      "source": [
        "# Russell and Norvig, AIMA Chapter 7 “Logical Agents” (Sections 7.1-7.5)\n",
        "\n",
        "chapter7_1 = \"\"\" A generic knowledge-based agent, when given a precept, adds it ot the knowledge base, then asks the knowledge base for the best action given this new\n",
        "information, and then updates the knowledge-based agent after the action has occured, i.e.:\n",
        "<table>\n",
        "<tr><td><b>function</b> <i>KB-AGENT(percept)</i> <b>returns</b> an <i>action</i><br />   <b>persistent</b>: <i>KB</i>, a knowledge base and <i>t</i>, a counter, initially 0,\n",
        "indicating time</td></tr>\n",
        "<tr><td><i>TELL(MAKE-PRECEPT-SENTENCE(precept,t))</i></td></tr>\n",
        "<tr><td><i>action</i> <- <i>ASK(KB, MAKE-ACTION-QUERY(t))</i></td></tr>\n",
        "<tr><td><i>TELL(KB,MAKE-ACTION-SENTENCE(action,t))</i></td></tr>\n",
        "<tr><td><i>t</i> <- <i>t</i> + 1</td></tr>\n",
        "<tr><td><b>return</b> <i>action</i></td></tr>\n",
        "</table>\n",
        "Here, <b>propositional logic</b> is used instead of the more expressive but computationally expensive <b>first-order logic</b>. The central component of any knowledge-based\n",
        "agent would be its <b>knowledge base</b>, hereafter KB. The KB consists of a series of <b>sentences</b> (which are related but not the same as sentence sin natural language).\n",
        "These sentences are expressed in <b>knowledge representation language</b> to assert some truths about the environment they exist in - sentences given as true initially\n",
        "and taken for granted are called <b>axioms</b>. New sentences can be added via the <i>TELL</i> and <i>ASK</i> operations. These both involve <b>inference</b> or the process\n",
        "of deriving new sentences from existing sentences.\n",
        "<br /><br />\n",
        "The detials of the representation langauge are hidden inside the functions that implement the interface between both the sensors and actuators on one end and the reasoning\n",
        "system and its core representation on the other:\n",
        "<ol><li><i>MAKE-PERCEPT-SEQUENCE</i> constructs a setence that asserts the given precept into the KB.</li>\n",
        "<li><i>MAKE-PERCEPT-SEQUENCE</i> constructs a sentence indicating the best action based on the current KB.</li>\n",
        "<li><i>MAKE-ACTION-SENTENCe</i> constructs a sentence that asserts the chosen action was taken.</li></ol>\n",
        "The agent and its goals are defined at the <b>knowledge level</b>, at which the agent should be able to simply state what it knows and be provided with the best action to take.\n",
        "The specifics of how these are calculated <b>implementation level</b>, e.g. via string manipulation or propagating signals down a neural network, should be abstracted away\n",
        "from the program at the knowledge level.\n",
        "<br /><br />\n",
        "There was originally some debate between the <b>declarative</b> approach, in which general sentences or truths are fed to a model which can then predicate actions, and\n",
        "the <b>procedural</b> approach, which involves explicitly encoding desired behaviors. But now we understand that the best performing agents will incorporate elements of both \n",
        "and that declarative knowledge can often be compiled into the underlying procedural code to improve performance.\"\"\"\n",
        "chapter7_2 = \"\"\" The <b>wumpus world</b> is a cave with several different passageways. The agent's goal is to traverse the cave and find the gold without falling into a pit \n",
        "or being eaten alive by the wumpus, either of which will result in the agent's death. The agent also has a bow and arrow it can use once to attempt to eliminate the wumpus\n",
        "and several sensors which help it identify potential scenarios. The task environment can be defined precisely via the following PEAS description:\n",
        "<ul><li><b>Performance measure:</b> The agent is given +1000 points for climbing out of the cave with the gold, -1 point for each action taken, -10 points to use the arrow,\n",
        "and -1000 points for dying - either by being eaten by the wumpus or falling into the pit.</li>\n",
        "<li><b>Environment:</b> The default environment is a closed 4x4 grid, surrounded by walls. The gold and wumpus are placed randomly from a uniform distribution at any of the\n",
        "other squares than [1,1], where the agent is placed. Every other square can also be a pit, with probability 0.2.</li> \n",
        "<li><b>Actuators:</b>Tgere are several actions that the agent can make:\n",
        "<ul><li><i>Forward</i>: If an agent bumps into a wall, nothing happens.</li>\n",
        "<li><i>TurnLeft</i> and <i>TurnRight</i> by 90 degrees.</li>\n",
        "<li><i>Grab</i>: The agent picks up the gold, can only be used when the agent is in the same square as the gold.</li>\n",
        "<li><i>Shoot</i>: Fires an arrow in the direction the agent is facing. The arrow continues a straight line until it encounters a wall or the wumpus. If it hits the wumpus,\n",
        "it kills the wumpus, which lets out a <i>Scream</i>. The agent can now travel to that square without being eaten.</li>\n",
        "<li><i>Climb</i>: The agent can climb out of the cave, can only be used at square [1,1].</li></ul></li>\n",
        "<li><b>Sensors:</b> For each of the five sensors, the agent will perceive a:\n",
        "<ul><li><i>Stench</i> in squares that are directly but <i>not</i> diagonally adjacent to the wumpus.</li>\n",
        "<li><i>Breeze</i> in squares that are directly adjacent to a pit.</li>\n",
        "<li><i>Glitter</i> in a square where there is gold.</li>\n",
        "<li><i>Bump</i> if the agent walks into a wall.</li>\n",
        "<li><i>Scream</i> that can be heard anywhere in the cave when a wumpus dies.</li></ul>\n",
        "</li></ul>\n",
        "The wumpus environment is clearly deterministic, discrete, single-agent, and static, since the wumpus does not move. It is also a sequential environment since rewards are\n",
        "contingent on moves being taken and is partially observable, because only a few aspects, e.g. the agent's location, the wumpus' health, if the bow and arrow have been used,\n",
        "are known at any given time. As the agent goes through the board, it makes conclusions based on previous observations to speculate on the location of pits and the wumpus,\n",
        "for example. Despite this, 21% of all possible environments are utterly unfair, e.g. the gold is located in a pit, which forces the agent to choose to go home empty-handed.\n",
        "The conclusion is <i>guaranteed</i> to be correct if the provided information is correct.\"\"\"\n",
        "chapter7_3 = \"\"\" Logic has certain fundamental characteristics regardless of its form. These include its <b>syntax</b> and <b>semantics</b>. Together, these define the\n",
        "<b>truth</b> of each statement in each <b>possible world</b> or <b>model</b>. We say that if a sentence <i>alpha</i> is true in a model <i>m</i> that <i>m</i> <b>satisfies</b>\n",
        "<i>alpha</i> or alternatively, that <i>m</i> <b>is a model of</b> <i>alpha</i>.\n",
        "<br /><br />\n",
        "The process of <b>entailment</b> involves seeing if a sentence <i>follows logically</i> from another sentence. <i>alpha</i> entails <i>beta</i> if and only if for every\n",
        "sentence in which <i>alpha</i> is true, <i>beta</i> is also true, i.e. <i>M(alpha)</i> is a proper subset of <i>M(beta)</i>. This is similar to how if<i>y = 0</i> then \n",
        "<i>yz = 0</i>. In the Wumpus World example, if the agent does not sense a breeze in [1,1], then it follows that there is no pit in [1,2] or [2,1], so the KB entails the \n",
        "statement <i>alpha<sub>1</sub></i> = \"There is no pit in [1,2]\" but not the statement <i>alpha<sub>2</sub></i> = \"There is no pit in [2,2]\" as this has not been yet shown. \n",
        "This also illustrates the process of <b>logical inference</b> or applying the definition of entailment using newly found information in the KB to derive new conclusions. \n",
        "In particular, this illustrates the <b>model checking</b> algorithm, which enumerates all the possibilities of a KB to see if <i>M(KB)</i> is a proper subset of <i>M(a)</i>.\n",
        "<br /><br />\n",
        "One important consideration is that any model used should be both <b>sound</b> or <b>truth preserving</b> and also <b>complete</b>. This means that if the sentences in the\n",
        "initial KB are true, any of the derived sentences maintain the truth properties of the other sentences in the KB and the inference algorithm proceeds by deriving all\n",
        "sentences that follow from the KB in each iteration. It follows from this that any derived sentences correspond to real-world relationships and imply the truthness of other\n",
        "properties in the real world.\n",
        "<br /><br />\n",
        "Another consideration is <b>grounding</b> - i.e. how we know that statements in the KB are true in the world. The soundness of these statements come from the fact that\n",
        "statements added to the KB come from the process of sensing in the agent. While there is not a direct representation always for derived sentences, these come through\n",
        "a process of <b>learning</b>, which can be fallible but one should still be optimistic given a good learning procedure.\"\"\"\n",
        "chapter7_4 = \"\"\"<b>Propositional logic</b> involves representing <b>atomic sentences</b> by a single <b>propositional symbol</b>. From this, <b>complex sentences</b>\n",
        "can be constructed by stringing together atomic statements with <b>logical connectives</b>. These are:\n",
        "<ul><li>~ (not): the <b>negation</b> returns the opposite truth value of the atomic sentence it is negating. A <b>literal</b> in this context can either be a\n",
        "<b>positive literal</b> (an atomic statement) or a <b>negative literal</b> (a negated atomic statement).</li>\n",
        "<li>/\\ (and): the <b>conjunction</b> has parts that are referred to as <b>conjuncts</b>. It is <i>true</i> if and only if all the conjuncts are true.</li>\n",
        "<li>\\/ (or): the <b>disjunction</b> has parts that are referred to as <b>disjuncts</b>. If either (or both) of the disjuncts are <i>true</i>, then so is the disjunction.\n",
        "This is notably different from the colloquial \"or\" which means \"one or the other\" (in logic, this is the \"exclusive or\" or (xor) operator).</li>\n",
        "<li>-> (implies): the <b>implication</b> can be broken into two parts: the <b>antecedent</b> or <b>premise</b> and the <b>consequent</b> or <b>conclusion</b>. It is\n",
        "<i>true</i> when the premise and conclusion are both <i>true</i> or when the premise is <i>false</i>, but is <i>false</i> when the premise is <i>true</i> but the\n",
        "conclusion is <i>false</i>. When the premise is <i>false</i>, the implication is <i>vacuously true</i> as the implication makes no claim about the environment\n",
        "when the premise is <i>false</i>.</li>\n",
        "<li><-> (if and only if): the <b>biconditional</b> is only <i>true</i> if <i>both</i> statements are true.</li></ul>\n",
        "The BNF (Backus-Naur) Form states the grammar of sentences in propositional logic, along with operator precedences.\n",
        "<i><table>\n",
        "<tr><td>Sentence</td><td>-> AtomicSentence | ComplexSentence</td></tr>\n",
        "<tr><td>AtomicSentence</td><td>-> True | False | P | Q | R | ...</td></tr>\n",
        "<tr><td>ComplexSentence</td><td>-> ( Sentence) )</td><tr>\n",
        "<tr><td></td><td>|  ~Sentence</td></tr>\n",
        "<tr><td></td><td>|  Sentence /\\ Sentence</td></tr>\n",
        "<tr><td></td><td>|  Sentence \\/ Sentence</td></tr>\n",
        "<tr><td></td><td>|  Sentence -> Sentence</td></tr>\n",
        "<tr><td></td><td>|  Sentence <-> Sentence</td></tr>\n",
        "<tr><td>Operator Precedence</td><td>~, /\\, \\/, ->, <-></td></tr>\n",
        "</table>\n",
        "</i>\n",
        "Complex sentences may be evaluated with <b>truth tables</b>. These expand all the possibilities for truth values from a component, e.g. given a statement not \n",
        "<i>P<sub>1</sub></i> /\\ (<i>P<sub>2</sub></i> \\/ <i>P<sub>3</sub></i>), it is known that if the statements are <i>true</i>, <i>false</i>, and <i>true</i> respectively\n",
        "then the statement evaluates to <i>true</i> /\\ (<i>false</i> \\/ <i>true</i>) = <i>true</i> /\\ <i>true</i> = <i>true</i>.\n",
        "<br /><br />\n",
        "An inference algorithm should be implemented to be both <b>sound</b> and <b>complete</b>, meaning that everything expanded is entailed by the KB (thereby, preserving\n",
        "the truthfulness of the KB) and that it expands all the possibilities for a given KB.\"\"\"\n",
        "chapter7_5 = \"\"\"<b>Theorem proving</b> or applying rules of inference directly to sentences is oftentimes more efficient than model checking. Three related key concepts:\n",
        "<ul><li><b>logical equivalence</b>: sentences are logically equivalent if they are true in the same set of models.</li>\n",
        "<li><b>validity</b>: true in <i>all</i> models, also the same as a <i>tautology</i></li>\n",
        "<li><b>satisfiability</b>: if a statement is satisfied by or true in <i>some</i> model</li>\n",
        "</ul>\n",
        "It can be said that a sentence is viable if and only if its negation is unsatisfiable and vice versa. This is also a form of proof by <b>refutation</b> or <b>contradiction</b>.\n",
        "<br /><br />\n",
        "Some <b>inference rules</b> that can be applied to derivve a proof include <b>Modus ponus</b> or inferring the implication if its condition is met, the <b>And-elimination</b>\n",
        "which states any of the conjuncts can be inferred, etc. <b>Resolution</b> is a rule that concludes a statement based on the negation of its other dijuncts.\n",
        "<br /><br />\n",
        "To get a sentence into <b>conjunctive normal form</b> (<b>CNF</b>) we do the following:\n",
        "<ol><li>Eliminate <->, replacing <i>A <-> B</i> with (<i>A -> B</i>) /\\ (<i>B -> A</i>)</li>\n",
        "<li>Eliminate ->, replacing <i>A -> B</i> with <i>~A</i> \\/ <i>B</i></li>\n",
        "<li>~ can only apply in literals, \"move ~ inwards\" by using De Morgan and double-negation elimination</li>\n",
        "<li>Distribute \\/ over /\\ wherever possible</li></ol>\n",
        "<b>Resolution closure</b> <i>RC(s)</i> is the set of all caluses derivable by a repeated application of resolution rule to the caluses in a set <i>S</i>. The completeness\n",
        "for resolution in propositional logic is via the <b>ground resolution theorem</b>: If a set of clauses is unsatisfiable, then the resolution closure of those clauses\n",
        "contains the empty clause.\n",
        "<br /><br />\n",
        "Sometimes, the full power of resolution is not needed. Some KBs have restrictions which enable them to use a more efficient inference algorithm. For example there are\n",
        "<b>definite clause</b>s which are a dijunction of literals in which <i>exactly one</i> is positive. Even more general is the <b>horn clause</b> which is a dijunction in\n",
        "which <i>at most</i> one is <i>positive</i> - these include <b>goal caluses</b> in which none are true. Another is the <i>k</i>-CNF sentence which is a CNF sentence with\n",
        "at most <i>k</i> literals. These are interesting because: \n",
        "<br /><br />\n",
        "The process of <b>forward chaining</b> is <b>data driven</b> because it begins from known facts and then adds statements until nof urther inferences can be made. It is both\n",
        "<b>sound since each inference is an application of Modus Ponus but also <b>complete</b> since if there were other sentences that could be added but were not, the\n",
        "algorithm would add those before terminating. Backwards chaining similarly works backwards from the query to find the implications whose conclusion is <i>q</i>. This is a\n",
        "form of <b>goal-directed reasoning</b> and is often linear with size of the KB since it only touches relevant facts.\"\"\"\n",
        "\n",
        "# Lecture notes \n",
        "module7_lecture_notes = \"\"\" Your notes here \"\"\"\n",
        "\n",
        "print(\"The length in words is\", len(chapter7_5.split(' ')))\n",
        "print(\"The length in characters is\", len(chapter7_5))\n",
        "\n",
        "# Definitions\n",
        "module7_definitions = {\n",
        "\"knowledge base\" : \"a set of sentences that comprise the central component of a knowledge-based agent\",\n",
        "\"sentence\" : \"a component statement of a knowledge base, expressed in knowledge rperesentation language\",\n",
        "\"knowledge representation language\" : \"a logical expression that represents some assertion about the real world in a knowledge base\",\n",
        "\"axiom\" : \"a sentence that is given as truthful, rather than being derived from other sentences\",\n",
        "\"inference\" : \"the process of deriving a new sentence based on old sentences\",    \n",
        "\"background knowledge\" : \"knowledge that is known beforehand\",\n",
        "\"knowledge level\" : \"the most abstract level in which the agent and its goals are defined\",\n",
        "\"implementation level\" : \"the specifics to how an agent is implemented\",\n",
        "\"declarative approach\" : \"giving an agent known statements until the agent knows how to behave\",\n",
        "\"procedural approach\" : \"explicitly defining how ana gent should behave\",\n",
        "\"performance measure\" : \"a function that defines how an agent has performed at a task\",\n",
        "\"environment\" : \"the available space for the agent, e.g. size of the model, placement of objects, etc.\",\n",
        "\"actuators\" : \"behaviors that an agent can perform\",\n",
        "\"sensors\" : \"precepts that are given to an agent to infer more about its environment\",\n",
        "\"syntax\" : \"the rules that define how sentences can be formed in a representation language\",\n",
        "\"semantics\" : \"meaning of a sentence, defines its truth value in a possible world\",\n",
        "\"model\" : \"a mathematical abstraction with a fixed truth value\",\n",
        "\"entailment\" : \"a sentence entails another if the other statement logically follows from it\", \n",
        "\"logical inference\" : \"applying entailments to derive new conclusions\",\n",
        "\"model checking\" : \"a specific logical inference algorithm that enumerates all possible models to validate the truthness of a sentence\",\n",
        "\"grounding\" : \"the connection between sentences and the real world\",\n",
        "\"soundness / truth preserving\" : \"entailed sentences that are derived do not invalidate the truthness of other senteces in knowledge base\",\n",
        "\"completness\" : \"inference algorithm derives all entailed sentences\" ,\n",
        "\"learning\" : \"the process constructing general rules from existing sentences\",\n",
        "\"propositional logic\" : \"logic in which atominc sentences are denoted by a single propositional symbol\",\n",
        "\"atomic sentence\" : \"a statement or claim about the world\",\n",
        "\"complex sentence\" : \"a sentence that combines multiple atomic sentences\",\n",
        "\"logical connectives\" : \"operators that connect different atomic sentences\",\n",
        "\"literal\" : \"an atomic sentence (a positive literal) or a negated atomic sentence (a negative literal\",\n",
        "\"negation\" : \"the inverse value of\",\n",
        "\"conjunction\" : \"logical and\",\n",
        "\"conjuncts\" : \"the components in a conjunction\",\n",
        "\"disjunction\" : \"logical or\",\n",
        "\"disjuncts\" : \"the components in a disjunction\",\n",
        "\"implication\" : \"an atomic sentence that follows from another atomic sentence\",\n",
        "\"premise\" : \"or antecedent, the first part of an implication or biconditional\",\n",
        "\"conclusion\" : \"or consequent, the implied part of an implication\",\n",
        "\"biconditional\" : \"logical if and only if\",\n",
        "\"truth value\" : \"if an atomic sentence is <i>true</i> or <i>false</i>\",\n",
        "\"truth tables\" : \"an expression of propositional logic rules and their truth values based on the atomic statements\",\n",
        "\"sound\" : \"an algorithm that implements the definition of entailment directly\",\n",
        "\"complete\" : \"an algorithm that derives every possible statement in the KB\"\n",
        "}\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The length in words is 430\n",
            "The length in characters is 2988\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "cellView": "form",
        "id": "lXNcHuKjyNXa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 3345
        },
        "outputId": "1e321781-cb1e-446e-a667-022868a508c7"
      },
      "source": [
        "#@title Preview of your summary\n",
        "#@markdown You don't need to modify anything in this cell. Just press play.\n",
        "module7_sections = [\n",
        "  (\"AIMA Chapter 7.1 Knowledge-Based Agents\", chapter7_1),\n",
        "  (\"AIMA Chapter 7.2 The Wumpus World\", chapter7_2),\n",
        "  (\"AIMA Chapter 7.3 Logic\", chapter7_3),\n",
        "  (\"AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\", chapter7_4),\n",
        "  (\"AIMA 7.5 Propositional Theorem Proving\", chapter7_5),\n",
        "  (\"Module 7 Lecture Notes\", module7_lecture_notes),\n",
        "]\n",
        "\n",
        "module7_markdown = format_module(7, \"Logical Agents\", \n",
        "                         module7_sections, \n",
        "                         module7_definitions)\n",
        "display(Markdown(module7_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 7: Logical Agents\n## AIMA Chapter 7.1 Knowledge-Based Agents\n A generic knowledge-based agent, when given a precept, adds it ot the knowledge base, then asks the knowledge base for the best action given this new\ninformation, and then updates the knowledge-based agent after the action has occured, i.e.:\n<table>\n<tr><td><b>function</b> <i>KB-AGENT(percept)</i> <b>returns</b> an <i>action</i><br />   <b>persistent</b>: <i>KB</i>, a knowledge base and <i>t</i>, a counter, initially 0,\nindicating time</td></tr>\n<tr><td><i>TELL(MAKE-PRECEPT-SENTENCE(precept,t))</i></td></tr>\n<tr><td><i>action</i> <- <i>ASK(KB, MAKE-ACTION-QUERY(t))</i></td></tr>\n<tr><td><i>TELL(KB,MAKE-ACTION-SENTENCE(action,t))</i></td></tr>\n<tr><td><i>t</i> <- <i>t</i> + 1</td></tr>\n<tr><td><b>return</b> <i>action</i></td></tr>\n</table>\nHere, <b>propositional logic</b> is used instead of the more expressive but computationally expensive <b>first-order logic</b>. The central component of any knowledge-based\nagent would be its <b>knowledge base</b>, hereafter KB. The KB consists of a series of <b>sentences</b> (which are related but not the same as sentence sin natural language).\nThese sentences are expressed in <b>knowledge representation language</b> to assert some truths about the environment they exist in - sentences given as true initially\nand taken for granted are called <b>axioms</b>. New sentences can be added via the <i>TELL</i> and <i>ASK</i> operations. These both involve <b>inference</b> or the process\nof deriving new sentences from existing sentences.\n<br /><br />\nThe detials of the representation langauge are hidden inside the functions that implement the interface between both the sensors and actuators on one end and the reasoning\nsystem and its core representation on the other:\n<ol><li><i>MAKE-PERCEPT-SEQUENCE</i> constructs a setence that asserts the given precept into the KB.</li>\n<li><i>MAKE-PERCEPT-SEQUENCE</i> constructs a sentence indicating the best action based on the current KB.</li>\n<li><i>MAKE-ACTION-SENTENCe</i> constructs a sentence that asserts the chosen action was taken.</li></ol>\nThe agent and its goals are defined at the <b>knowledge level</b>, at which the agent should be able to simply state what it knows and be provided with the best action to take.\nThe specifics of how these are calculated <b>implementation level</b>, e.g. via string manipulation or propagating signals down a neural network, should be abstracted away\nfrom the program at the knowledge level.\n<br /><br />\nThere was originally some debate between the <b>declarative</b> approach, in which general sentences or truths are fed to a model which can then predicate actions, and\nthe <b>procedural</b> approach, which involves explicitly encoding desired behaviors. But now we understand that the best performing agents will incorporate elements of both \nand that declarative knowledge can often be compiled into the underlying procedural code to improve performance.\n\n## AIMA Chapter 7.2 The Wumpus World\n The <b>wumpus world</b> is a cave with several different passageways. The agent's goal is to traverse the cave and find the gold without falling into a pit \nor being eaten alive by the wumpus, either of which will result in the agent's death. The agent also has a bow and arrow it can use once to attempt to eliminate the wumpus\nand several sensors which help it identify potential scenarios. The task environment can be defined precisely via the following PEAS description:\n<ul><li><b>Performance measure:</b> The agent is given +1000 points for climbing out of the cave with the gold, -1 point for each action taken, -10 points to use the arrow,\nand -1000 points for dying - either by being eaten by the wumpus or falling into the pit.</li>\n<li><b>Environment:</b> The default environment is a closed 4x4 grid, surrounded by walls. The gold and wumpus are placed randomly from a uniform distribution at any of the\nother squares than [1,1], where the agent is placed. Every other square can also be a pit, with probability 0.2.</li> \n<li><b>Actuators:</b>Tgere are several actions that the agent can make:\n<ul><li><i>Forward</i>: If an agent bumps into a wall, nothing happens.</li>\n<li><i>TurnLeft</i> and <i>TurnRight</i> by 90 degrees.</li>\n<li><i>Grab</i>: The agent picks up the gold, can only be used when the agent is in the same square as the gold.</li>\n<li><i>Shoot</i>: Fires an arrow in the direction the agent is facing. The arrow continues a straight line until it encounters a wall or the wumpus. If it hits the wumpus,\nit kills the wumpus, which lets out a <i>Scream</i>. The agent can now travel to that square without being eaten.</li>\n<li><i>Climb</i>: The agent can climb out of the cave, can only be used at square [1,1].</li></ul></li>\n<li><b>Sensors:</b> For each of the five sensors, the agent will perceive a:\n<ul><li><i>Stench</i> in squares that are directly but <i>not</i> diagonally adjacent to the wumpus.</li>\n<li><i>Breeze</i> in squares that are directly adjacent to a pit.</li>\n<li><i>Glitter</i> in a square where there is gold.</li>\n<li><i>Bump</i> if the agent walks into a wall.</li>\n<li><i>Scream</i> that can be heard anywhere in the cave when a wumpus dies.</li></ul>\n</li></ul>\nThe wumpus environment is clearly deterministic, discrete, single-agent, and static, since the wumpus does not move. It is also a sequential environment since rewards are\ncontingent on moves being taken and is partially observable, because only a few aspects, e.g. the agent's location, the wumpus' health, if the bow and arrow have been used,\nare known at any given time. As the agent goes through the board, it makes conclusions based on previous observations to speculate on the location of pits and the wumpus,\nfor example. Despite this, 21% of all possible environments are utterly unfair, e.g. the gold is located in a pit, which forces the agent to choose to go home empty-handed.\nThe conclusion is <i>guaranteed</i> to be correct if the provided information is correct.\n\n## AIMA Chapter 7.3 Logic\n Logic has certain fundamental characteristics regardless of its form. These include its <b>syntax</b> and <b>semantics</b>. Together, these define the\n<b>truth</b> of each statement in each <b>possible world</b> or <b>model</b>. We say that if a sentence <i>alpha</i> is true in a model <i>m</i> that <i>m</i> <b>satisfies</b>\n<i>alpha</i> or alternatively, that <i>m</i> <b>is a model of</b> <i>alpha</i>.\n<br /><br />\nThe process of <b>entailment</b> involves seeing if a sentence <i>follows logically</i> from another sentence. <i>alpha</i> entails <i>beta</i> if and only if for every\nsentence in which <i>alpha</i> is true, <i>beta</i> is also true, i.e. <i>M(alpha)</i> is a proper subset of <i>M(beta)</i>. This is similar to how if<i>y = 0</i> then \n<i>yz = 0</i>. In the Wumpus World example, if the agent does not sense a breeze in [1,1], then it follows that there is no pit in [1,2] or [2,1], so the KB entails the \nstatement <i>alpha<sub>1</sub></i> = \"There is no pit in [1,2]\" but not the statement <i>alpha<sub>2</sub></i> = \"There is no pit in [2,2]\" as this has not been yet shown. \nThis also illustrates the process of <b>logical inference</b> or applying the definition of entailment using newly found information in the KB to derive new conclusions. \nIn particular, this illustrates the <b>model checking</b> algorithm, which enumerates all the possibilities of a KB to see if <i>M(KB)</i> is a proper subset of <i>M(a)</i>.\n<br /><br />\nOne important consideration is that any model used should be both <b>sound</b> or <b>truth preserving</b> and also <b>complete</b>. This means that if the sentences in the\ninitial KB are true, any of the derived sentences maintain the truth properties of the other sentences in the KB and the inference algorithm proceeds by deriving all\nsentences that follow from the KB in each iteration. It follows from this that any derived sentences correspond to real-world relationships and imply the truthness of other\nproperties in the real world.\n<br /><br />\nAnother consideration is <b>grounding</b> - i.e. how we know that statements in the KB are true in the world. The soundness of these statements come from the fact that\nstatements added to the KB come from the process of sensing in the agent. While there is not a direct representation always for derived sentences, these come through\na process of <b>learning</b>, which can be fallible but one should still be optimistic given a good learning procedure.\n\n## AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\n<b>Propositional logic</b> involves representing <b>atomic sentences</b> by a single <b>propositional symbol</b>. From this, <b>complex sentences</b>\ncan be constructed by stringing together atomic statements with <b>logical connectives</b>. These are:\n<ul><li>~ (not): the <b>negation</b> returns the opposite truth value of the atomic sentence it is negating. A <b>literal</b> in this context can either be a\n<b>positive literal</b> (an atomic statement) or a <b>negative literal</b> (a negated atomic statement).</li>\n<li>/\\ (and): the <b>conjunction</b> has parts that are referred to as <b>conjuncts</b>. It is <i>true</i> if and only if all the conjuncts are true.</li>\n<li>\\/ (or): the <b>disjunction</b> has parts that are referred to as <b>disjuncts</b>. If either (or both) of the disjuncts are <i>true</i>, then so is the disjunction.\nThis is notably different from the colloquial \"or\" which means \"one or the other\" (in logic, this is the \"exclusive or\" or (xor) operator).</li>\n<li>-> (implies): the <b>implication</b> can be broken into two parts: the <b>antecedent</b> or <b>premise</b> and the <b>consequent</b> or <b>conclusion</b>. It is\n<i>true</i> when the premise and conclusion are both <i>true</i> or when the premise is <i>false</i>, but is <i>false</i> when the premise is <i>true</i> but the\nconclusion is <i>false</i>. When the premise is <i>false</i>, the implication is <i>vacuously true</i> as the implication makes no claim about the environment\nwhen the premise is <i>false</i>.</li>\n<li><-> (if and only if): the <b>biconditional</b> is only <i>true</i> if <i>both</i> statements are true.</li></ul>\nThe BNF (Backus-Naur) Form states the grammar of sentences in propositional logic, along with operator precedences.\n<i><table>\n<tr><td>Sentence</td><td>-> AtomicSentence | ComplexSentence</td></tr>\n<tr><td>AtomicSentence</td><td>-> True | False | P | Q | R | ...</td></tr>\n<tr><td>ComplexSentence</td><td>-> ( Sentence) )</td><tr>\n<tr><td></td><td>|  ~Sentence</td></tr>\n<tr><td></td><td>|  Sentence /\\ Sentence</td></tr>\n<tr><td></td><td>|  Sentence \\/ Sentence</td></tr>\n<tr><td></td><td>|  Sentence -> Sentence</td></tr>\n<tr><td></td><td>|  Sentence <-> Sentence</td></tr>\n<tr><td>Operator Precedence</td><td>~, /\\, \\/, ->, <-></td></tr>\n</table>\n</i>\nComplex sentences may be evaluated with <b>truth tables</b>. These expand all the possibilities for truth values from a component, e.g. given a statement not \n<i>P<sub>1</sub></i> /\\ (<i>P<sub>2</sub></i> \\/ <i>P<sub>3</sub></i>), it is known that if the statements are <i>true</i>, <i>false</i>, and <i>true</i> respectively\nthen the statement evaluates to <i>true</i> /\\ (<i>false</i> \\/ <i>true</i>) = <i>true</i> /\\ <i>true</i> = <i>true</i>.\n<br /><br />\nAn inference algorithm should be implemented to be both <b>sound</b> and <b>complete</b>, meaning that everything expanded is entailed by the KB (thereby, preserving\nthe truthfulness of the KB) and that it expands all the possibilities for a given KB.\n\n## AIMA 7.5 Propositional Theorem Proving\n<b>Theorem proving</b> or applying rules of inference directly to sentences is oftentimes more efficient than model checking. Three related key concepts:\n<ul><li><b>logical equivalence</b>: sentences are logically equivalent if they are true in the same set of models.</li>\n<li><b>validity</b>: true in <i>all</i> models, also the same as a <i>tautology</i></li>\n<li><b>satisfiability</b>: if a statement is satisfied by or true in <i>some</i> model</li>\n</ul>\nIt can be said that a sentence is viable if and only if its negation is unsatisfiable and vice versa. This is also a form of proof by <b>refutation</b> or <b>contradiction</b>.\n<br /><br />\nSome <b>inference rules</b> that can be applied to derivve a proof include <b>Modus ponus</b> or inferring the implication if its condition is met, the <b>And-elimination</b>\nwhich states any of the conjuncts can be inferred, etc. <b>Resolution</b> is a rule that concludes a statement based on the negation of its other dijuncts.\n<br /><br />\nTo get a sentence into <b>conjunctive normal form</b> (<b>CNF</b>) we do the following:\n<ol><li>Eliminate <->, replacing <i>A <-> B</i> with (<i>A -> B</i>) /\\ (<i>B -> A</i>)</li>\n<li>Eliminate ->, replacing <i>A -> B</i> with <i>~A</i> \\/ <i>B</i></li>\n<li>~ can only apply in literals, \"move ~ inwards\" by using De Morgan and double-negation elimination</li>\n<li>Distribute \\/ over /\\ wherever possible</li></ol>\n<b>Resolution closure</b> <i>RC(s)</i> is the set of all caluses derivable by a repeated application of resolution rule to the caluses in a set <i>S</i>. The completeness\nfor resolution in propositional logic is via the <b>ground resolution theorem</b>: If a set of clauses is unsatisfiable, then the resolution closure of those clauses\ncontains the empty clause.\n<br /><br />\nSometimes, the full power of resolution is not needed. Some KBs have restrictions which enable them to use a more efficient inference algorithm. For example there are\n<b>definite clause</b>s which are a dijunction of literals in which <i>exactly one</i> is positive. Even more general is the <b>horn clause</b> which is a dijunction in\nwhich <i>at most</i> one is <i>positive</i> - these include <b>goal caluses</b> in which none are true. Another is the <i>k</i>-CNF sentence which is a CNF sentence with\nat most <i>k</i> literals. These are interesting because: \n<br /><br />\nThe process of <b>forward chaining</b> is <b>data driven</b> because it begins from known facts and then adds statements until nof urther inferences can be made. It is both\n<b>sound since each inference is an application of Modus Ponus but also <b>complete</b> since if there were other sentences that could be added but were not, the\nalgorithm would add those before terminating. Backwards chaining similarly works backwards from the query to find the implications whose conclusion is <i>q</i>. This is a\nform of <b>goal-directed reasoning</b> and is often linear with size of the KB since it only touches relevant facts.\n\n## Module 7 Lecture Notes\n Your notes here \n\n## Module 7 Definitions\n* **Knowledge base** - a set of sentences that comprise the central component of a knowledge-based agent\n* **Sentence** - a component statement of a knowledge base, expressed in knowledge rperesentation language\n* **Knowledge representation language** - a logical expression that represents some assertion about the real world in a knowledge base\n* **Axiom** - a sentence that is given as truthful, rather than being derived from other sentences\n* **Inference** - the process of deriving a new sentence based on old sentences\n* **Background knowledge** - knowledge that is known beforehand\n* **Knowledge level** - the most abstract level in which the agent and its goals are defined\n* **Implementation level** - the specifics to how an agent is implemented\n* **Declarative approach** - giving an agent known statements until the agent knows how to behave\n* **Procedural approach** - explicitly defining how ana gent should behave\n* **Performance measure** - a function that defines how an agent has performed at a task\n* **Environment** - the available space for the agent, e.g. size of the model, placement of objects, etc.\n* **Actuators** - behaviors that an agent can perform\n* **Sensors** - precepts that are given to an agent to infer more about its environment\n* **Syntax** - the rules that define how sentences can be formed in a representation language\n* **Semantics** - meaning of a sentence, defines its truth value in a possible world\n* **Model** - a mathematical abstraction with a fixed truth value\n* **Entailment** - a sentence entails another if the other statement logically follows from it\n* **Logical inference** - applying entailments to derive new conclusions\n* **Model checking** - a specific logical inference algorithm that enumerates all possible models to validate the truthness of a sentence\n* **Grounding** - the connection between sentences and the real world\n* **Soundness / truth preserving** - entailed sentences that are derived do not invalidate the truthness of other senteces in knowledge base\n* **Completness** - inference algorithm derives all entailed sentences\n* **Learning** - the process constructing general rules from existing sentences\n* **Propositional logic** - logic in which atominc sentences are denoted by a single propositional symbol\n* **Atomic sentence** - a statement or claim about the world\n* **Complex sentence** - a sentence that combines multiple atomic sentences\n* **Logical connectives** - operators that connect different atomic sentences\n* **Literal** - an atomic sentence (a positive literal) or a negated atomic sentence (a negative literal\n* **Negation** - the inverse value of\n* **Conjunction** - logical and\n* **Conjuncts** - the components in a conjunction\n* **Disjunction** - logical or\n* **Disjuncts** - the components in a disjunction\n* **Implication** - an atomic sentence that follows from another atomic sentence\n* **Premise** - or antecedent, the first part of an implication or biconditional\n* **Conclusion** - or consequent, the implied part of an implication\n* **Biconditional** - logical if and only if\n* **Truth value** - if an atomic sentence is <i>true</i> or <i>false</i>\n* **Truth tables** - an expression of propositional logic rules and their truth values based on the atomic statements\n* **Sound** - an algorithm that implements the definition of entailment directly\n* **Complete** - an algorithm that derives every possible statement in the KB\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vX7lbnJd4sT5"
      },
      "source": [
        "# Your Exam Notes\n",
        "\n",
        "Here are your full set of exam notes that you can use on your midterm.  Choose **Runtime > Run all**, then your notes will be displayed.\n",
        "\n",
        "If you are taking the exam in person, then you must print a copy of your notes for yourself.   Here's how to print:\n",
        "* Run the cell below. \n",
        "* Click its \"Mirror cell in tab\" button (this is located in the upper right corner of the cell, immediately to the right of the trash can).  This will open the cell.  \n",
        "* In Chrome, you can then print your notes using the **File > Print** option from the browser's menu.  \n",
        "* In the print options, change the layout from Landscape to Portrait. \n",
        "* Clicking the Save, will save a PDF that you can then print.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "T3cm1WV542_a",
        "outputId": "d12cba48-67eb-4527-b8aa-4a15383723c9"
      },
      "source": [
        "# This will generate the full set of notes that you can use for your exam.\n",
        "full_markdown = \"\\n\\n\".join([module1_markdown, \n",
        "                             module2_markdown, \n",
        "                             module3_markdown, \n",
        "                             module4_markdown, \n",
        "                             module5_markdown,\n",
        "                             module6_markdown,\n",
        "                             module7_markdown,]\n",
        "                       )\n",
        "display(Markdown(full_markdown))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/markdown": "# Module 1: Rational Agents and Task Environments\n## AIMA Chapter 1.1 What is AI?\n There is some debate over the meaning of being intelligent, leading to the following four approaches:\n<table>\n<tr><td></td><td><b>Thinking</b></td><td><b>Acting</b></td></tr>\n<tr>\n<td><b>Humanly</b></td>\n<td>We can model how humans think in three main ways:\n<ul><li><b>introspection</b> - monitoring our own thoughts</li>\n<li><b>psychological experiments</b> - observing people in action</li>\n<li><b>brain imaging</b> - observing the brain in action</li></ul>\nThese developments have led to fields like <b>cognitive science</b> which use experimental techniques from psychology, and some of these brain models have made their way into\ncomputer vision research.</td>\n<td>The <b>Turing test</b> asks if a machine can appear human to a human interrogator. In order to do this, a Turing machine would need the following abilities:\n<ul><li><b>natural language processing</b> to be able to communicate the way a human does</li>\n<li><b>knowledge representation</b> to store and reason about what it perceives</li>\n<li><b>automated reasoning</b> to respond to questions and come up with new conclusions</li>\n<li><b>machine learning</b> to extrapolate patterns to apply to new situations</ul>\nTo apss the more complicated <b>total Turing test</b> which requires interacting with people, it would also need:\n<ul><li><b>computer vision</b> to perceive objects in the real world</li>\n<li><b>robotics</b> to move and be able to manipulate objects</li></ul></td></tr>\n<tr>\n<td><b>Rationally</b></td>\n<td>The \"Laws of Thought\" approach is modeled after attempts to codify \"right thinking\" by philosophers like Aristotle. He defined certain <b>syllogisms</b> for\nlogical reasoning that formed the basis for the field of <b>logic</b>. In efforts since these are codified into a formal language via the <b>logicist</b> tradition.\nWhen outcomes cannot be defined with <i>certainty</i>, the theory of <b>probability</b> allows rigorous reasoning with information that is uncertain.</td>\n<td>An <b>agent</b> is something that acts. Computer argents are expected to operate autonomously by perceiving its\nenvironment, persisting over an extended time period, adapting to change, and identifying and pursuing goals. A <b>rational agent</b> is one\nthat acts in a way that achieves the best expected outcome when given an uncertain situation.This has the advantage that it is more general than the\n\"laws of thought\" approach as there are several ways to achieve rationality beyond inference and it is amenable to the scientific process as it can\nbe rigorously defined and generalized. Specific agent designs and goals are less ambiguous to define than mimicking human behavior.</td></tr>\n</table><br />\nAs AI becomes more complicated, the specific goals and objectives are more ambiguous to define. How should a self-driving car\nbehave to avoid endangering other errant drivers, passenger comfort, etc? These lead to the <b>value alignment problem</b> where we ask if the values\nin the AI match those of humans.\n\n## AIMA Chapter 27.1 The Limits of AI\nJohn Searle, a philosopher, proposed the concept of <b>strong AI</b> and <b>weak AI</b>, where the former denotes machines that are actually\ncapable of human thought and the altter referring to machines that merely act like humans. The notion of weak AI has since been disputed and indeed, Alan Turing anticipated\nseveral of the main arguments that have been made against AI:\n<ol>\n<li>Turing's \"argument from the informality of behavior\" stated that humans do not have a formal set of rules and instead use informal guidance to make decisions. This view\nwas further argued for by Dreyfus who wrote books on what AI couldn't do, but these models have since been discarded as the idea of <b>Good Old-Fashioned AI (GOFAI)</b>.\nIndeed, while the <b>qualification problem</b> has shown that some problems are difficult to explicitly model, deep learning and other techniques have proven to be able\nto structure informal situations. Another variant of this argument is <b>embodied cognition</b>: cognition takes place throughout the whole body, placed in an environment.\nThen, we only need to study the agent's behavior as a whole - under such a view, robotics, vision, etc. become central as opposed to peripheral.</li>\n<li>The \"argument from disability\" states that a machine can never do certain things - Turing listed \"tell right from wrong, make mistakes, fall in love, enjoy strawberries\nand cream, make someone fall in love with it, learn from experience, etc.\" Computers have shown to be able to do many of these things, including making significant\ncontributions to fields like chemistry, computer science, even creating new art via style transfer.</li>\n<li>Gobel stated that for any axiomatic framework <i>F</i> that is capable of doing arithmetic, one can construct\na Godel sentence <i>G(F)</i> which ahd two properties:\n<ul><li><i>G(F)</i> is a sentence of <i>F</i> that cannot be proved within <i>F</i>\n<li>if <i>F</i> is consistent, then it follows that <i>G(F)</i> is true.</li></ul>\nIt follows from this that by the incompleteness theorem, machines are limited and cannot establish their own Godel sentence. However, this has been contended with for a couple\nof reasons:\n<ol><li>Just because one agent, e.g. human cannot assert something, that does not mean other people or machines can</li>\n<li>Godel's incompleteness theorem apply to <i>mathematics</i> and not necessarily <i>computers</i> as no one is capable of proving something that is impossible to prove.\nHumans sometimes assume their own consistency only to be proven false later, as in the four-color map problem.</li>\n<li>This fails to consider finite limitations for computers and human brains, who can be described as very large propositional logic systems. But just as a human can change\nits decision, a computer may have different results with machine learning or software rewriting.</li></ol>\n</ol>\nThe <b>turing test</b> has not been of much focus in the AI community although some Chatbots have shown a lot of potential.\n\n\n## AIMA Chapter 27.1 Can Machines Really Think?\nSome further debate has gone into whether it is even important that machines think - for example, submarines don't necessarily swim and airplanes don't\nnecessarily fly. Turing himself suggested the notion of <b>polite convention</b> - that is, we take what other people say at face value and don't try to break apart their\ninner thoughts or how they reason about what they say, so similar leniencey should be given to machines. There have been some philosophers who disagree with this - Searle\nalso came up with a thoguht experiment called the <b>Chinese room</b>, in which a translator looks up every single English word that comes in and presses a button resulting\nin a correct Chinese translation on the other end. Searle argues that the translator does not in fact know Chinese even though the end user would be getting perfect\nChinese translations. Searle was thus a big proponent of <b>biological naturalism</b> which states that mental states are caused by low-level biological processes in the\nneurons themselves and the intrinsic properties of these are what define intelligence. Thus much of this debate surrounds the concept of <b>consciousness</b> or awareness\nof the outside world, and the subjective experience of living, e.g. the concept of self. The <b>qualia</b> or intrinsic nature of these experiences is not so important\naccording to Turing, and further some of these aspects can be programmed into machines but this <i>awareness</i> is still mostly a part of the human experience.\n\n## AIMA 2.1 Agents and Environments\nHere, an <b>agent</b> is any object that is capable of perceiving its <b>environment</b> with the use of <b>sensors</b> and then acting on these <b>percepts</b>\nusing <b>actuators</b>. Some examples of agents include a piece of software which receives packets and I/O from a user and then sends packets or displays information in\nresponse, or a robot that perceives through its camera lens and infrared range finders and then moves itself based on the environment it perceives. In general, an agent\nconsiders its entire <b>percept sequence</b> and the agent's choice of what action to take at any given instant depends on any built-in knowledge provided and on the\npercept sequence up to the current moment, and not on anything it has yet to perceive. The way that an agent behaves can be described by its <b>agent function</b> which maps\na percept sequence to a specific action.\n<br /><br />\nOne interpretation of the agent function is that it <i>tabulates</i>the agent function - but such a table could be infinitely large and this is an <i>external</i> representation\nof the agent's choices. <i>Internally</i>, the agent function is implemented by the <b>agent program</b>. The key difference between the two is that the agent function is an\nabstract mathematical function while the agent program is its actual implementation.\n<br /><br />\nA final note is that the idea of an agent is meant to be a paradigm or tool for analyzing a system as opposed to an absolute characterization of the system. There is no\ninherent dichotomy between agents and non-agents in the real world. In some sense, most of engineering involves artifacts that interact the world and artifical intelligence\noperates at the end of the spectrum where resources have significant computational resources and are tasked with making nontrivial decisions.\n\n## AIMA 2.2 Good Behavior: The Concept of Rationality\nA <b>rational agent</b> can be defined as one that chooses the right action. But in terms of what defines \"right\", AI tends to stick to the notion of\n<b>consequentialism</b>, or evaluating its success by the consequences of its actions. The <b>performance measure</b> is a function that evaluates any given sequence of successive\nenvironment states. Since machines do not have desires or preferences, at least not initially, it is better in general to design performance measures based on the outcome one wants to achieve in an environment (the goal state) as opposed to how one thinks \nthat an agent <i>should</i> behave.\n<br /><br />\nThe rational behavior at any given time depends on the following four factors:\n<ol><li>The performance measure that defines the criterion of a succesful outcome</li>\n<li>Any prior knowledge the agent has about its environment</li>\n<li>The possible actions that an agent can take</li>\n<li>The percept sequence to date that the agent has</li></ol>\nThe <b>definition of a rational agent</b> is that for any given possible percept sequence, a rational agent is one that will select an action that is expected to maximize\nits performance measure using its built-in knowledge and evidence obtained from the percept sequence so far. An example of this is the vaccum cleaner:\n<ol><li>The performance measure awards one point for each clean square over a \"lifetime\" of 1000 steps.</li>\n<li>The \"geography\" or overlal dimensions of the space are known <i>a priori</i> but the dirt distribution and locations are not.</li>\n<li>The possible actions are <i>Right</i>, <i>Left</i>, and <i>Suck</i>. The former two move the agent one square in the given direction except when doing so takes the\nagent outside of its environment; it stays in place instead.</li>\n<li>The agent correctly perceives its location and if a square has dirt.</li></ol>\nOne should be careful not to confuse rationality with <b>omniscience</b> or the state of knowing everything. Rationality aims to maximize the <i>expected</i> performance\nand depends only on the percept sequence <i>to date</i>, while perfection would maximize the <i>actual</i> performance. If some freak accident were to happen later on in the\npercept sequence, this is not necessarily known to a rational agent. This does not mean, however, that a rational agent should be not encouraged to engage in directly\nunintelligent activities. A rational agent is still expected to do actions leading to <b>information gathering</b> as in getting the best information possible to modify future\npercepts - like looking both ways before crossing the street. A rational agent can thus be expected to <b>explore</b> but also to <b>learn</b> as much as possible from what\nit perceives. There is an expectation that a rational agent can behave <b>autonomously</b>, that is, it is able to independently figure out how to compensate for whatever\nlack of knowledge it has initially by learning as much as possible from its percept sequence.\n\n\n## AIMA 2.3 The Nature of Environments\n Task environments can be characterized by several dimensions which influence how agents interact. These are:\n<ul>\n<li><b>Fully</b> vs. <b>partially</b> observable: Is the agent capable of perceiving the entire environment at any given time? An environment is fully observable if sensors are\nable to perceive all <i>relevant</i> information based on their performance measure. Oftentimes, environments are partially observable because of noise or other limitations\nthat restrict the agent's sensors. If there are no sensors at all the environment is <b>unobservable</b>.</li>\n<li><b>Single-agent</b> versus <b>multiagent</b>: This relates to if there is a single agent playing a game or there are other agents involved. There are some subtle issues\nto consider: how do agents treat each other for example? In a <b>competitive</b> environment like chess, they may be trying to minimize the performance measure of other agents\nwhile in a <b>cooperative</b> setting like self-driving cars they may try to maximize each other's performance measure.</li>\n<li><b>Deterministic</b> and <b>nondeterministic</b>:A deterministic task environment is one where the next state is dictated completely by the current state and the action\nthat the agent takes. Many partially observable environments appear to be nondeterministic, as are many real-world situations. <b>Stochastic</b> is another word that is used\nfor nondeterministic, but refers to specific environments where outcomes are dictated by chance or probability.</li>\n<li><b>Episodic</b> vs. <b>sequential</b>: In an episodic task environment, each exerpeience can be segmented into distinct episodes, e.g. most classification tasks. If an \nassembly line identifies one product as defective, this has no effect on how it categorizes the next. Conversely, sequential environments are ones where the current decision\naffects all future decisions; e.g. a move taken in a game of chess affects all future moves.</li>\n<li><b>Static</b> and <b>dynamic</b>: An environment that does not change while the agent makes its choice is static, otherwise the task environment is dynamic. Static\nenvironments are simpler to perceive as the agent is not restricted by an amount of time to make a decision. Dynamic environments require the agent to constantly re-evaluate\nits environment. There are also <b>semidynamic</b> environments like chess where the chessboard does not change but there is a penalty associated with deciding a move.</li>\n<li><b>Discrete</b> vs. <b>continuous</b>: These relate to the <i>state</i> of the environment, the granularity at which <i>time</i> is handled and both the <i>percepts</i>\nand <i>actions</i> of agents are described. In a continuous environment, there are a range of values that change smoothly over time while discrete environments are fixed.</li>\n<li><b>Known</b> vs. <b>unknown</b>: In the former, the outcome (or probabilities) of all actions are given; otherwise, the agent will have to learn these.</li>\n</ul>\n\n## AIMA 2.4 The Structure of Agents\n\nThere are four basic types of systems underlying all intelligent systems:\n<ol><li>A <b>simple reflex agent</b> selects actions based on its <i>current</i> precept, ignoring the rest of its history. It follows a <b>condition-action role</b> which is\na statement of the form <b>if</b> <i>x</i> <b>then</b> <i>y</i>. This works best only in scenarios where the correct decision can be made on the basis of the current precept,\ni.e. a fully observable environment. Also can lead to infinite loops - <b>randomization</b> can escape\nloops but more sophisticated agents can handle better</li>\n<li>A model-based reflex agent can keep track of the part of the world it can no longer see by maintaining an <b>internal state</b> which holds its precept's memory. This\nrequires knowledge about the world in the <b>transition model</b> and information about how the state of the world is affected by actions in the <b>sensor model</b>. An agent\nthat uses such models together is a <b>model-based agent</b></li> \n<li><b>Goal based agents</b> know about the <b>goal</b> information or what state is ultimateily desirable. Through methods like <b>search</b> or <b>planning</b> they exhaust\ndifferent possibilities, checking after to see ifthe goal-state has been reached. This is more complicated than the previous two because it considers not just the immediate\nconsequence of an action but whether that action benefits the agent or not.</li>\n<li><b>Utility-based agents</b> evaluate different states to see how much <b>utility</b> they generate for an agent. They evaluate this using the <b>utility function</b>,\nan internalization of the performance measure. Since this is not known always, these maximize the <b>expected utility</b> of outcomes - the value of\nall the outcomes weighted by their probability. In some cases, a <b>model-free agent</b> can decide what action based on utility without storing a model.</li>\n</ol>\n<b>Learning agents</b> use a distinct <b>learning element</b> to improve and a <b>performance element</b> for selecting external actions. The former takes feedback\nfrom the <b>critic</b> and decides how to perform in the future. It uses a <b>problem generator</b> to expand possibilities.\nRepresentations can be structured as:\n<ul><li>in an <b>atomic representation</b> , each state of the world is indivisible - there is no internal structure, just a black box, e.g. MDP</li>\n<li>a <b>factored representation</b> breaks fixed state into <b>variables</b> or <b>attributes</b> which each have a <b>value</b></li>\n<li>to capture relationship, need a <b>structured rpresentation</b> in which objects have relationships that can be described. also of note is concept of <b>expressiveness</b>:\na more expressive representation can capture everything a less expressive one can</li>\n<li><b>localist representation</b> implies physical mapping of concept to memory</li>\n<li>a <b>distributed representation</b> involves multiple memory locations for multiple different concepts</li></ul>\n\n## Module 1 Lecture Notes\n Your summary here \n\n## Module 1 Definitions\n* **Agent** - anything that can perceive and act in an environment\n* **Actuators** - devices that allow agent to act on information from sensors\n* **Environment** - the surroundings of an agent and everything it interacts with\n* **Sensor** - a device that can pick up a precept\n* **Percept** - the content being perceived by an agent\n* **Percept sequence** - the complete history of all the percepts perceived by an agent\n* **Agent function** - a function that maps any percept sequence to a respective action for the agent\n* **Agent program** - a program that implements the agent function\n* **Rational agent** - an agent that takes the ractional action\n* **Consequentialism** - a notion in philosophy that judges agents by the consequences of their actions\n* **Performance measure** - a function that evaluates a sequence of states in the environment\n* **Explore** - the process of gathering percepts in an unknown environment\n* **Information gathering** - the decision to do actions in order to modify future percepts\n* **Learn** - to gain new knowledge about an environment based on percepts\n* **Autonomy** - the ability to learn from percepts to compensate for incomplete or partial initial knowledge\n\n\n# Module 2: Python\n## Module 2 Lecture Notes\n Your summary here \n\n## Module 2 Definitions\n* **Mutability** - your definition here\n\n\n# Module 3: Search Problems\n## AIMA Chapter 3.1 Problem-Solving Agents\nAn agent that is capable of planning ahead and considering sequences of actions to find a potential solution that reaches a certain desired goal state can be\ndescribed as a <b>problem-solving agent</b>. It undertakes the process of <b>search</b> that can be either <b>informed</b> or <b>uninformed</b> depending on if the estimate\nto the goal state is known. If the situation is completely <b>unknown</b> then an agent can do no better at random guessing, but as information becomes available, the agent\nis capable of the following:\n<ul><li><b>Goal formation</b>: The agent defines what its primary <b>goal</b> is. This is important as it dictates and organizes the agent's future behavior and restricts\nwhich potential moves may be considered.</li>\n<li><b>Problem formulation</b>: The agent comes up with a description of the different states in the environment and the actions that are needed to make the goal, i.e.\nan abstract representation of the world.</li>\n<li><b>Search</b>: The agent will simulate multiple different paths to see if it can find a <b>solution</b>, or a sequence of actions that lead to goal state, if one exists.\n</li><li><b>Execution</b>: The agent evaluates the actions from its solution, one at a time.</li></ul>\nThe environment can be further characterized as <b>open-loop</b> or <b>closed-loop</b> if the environment is nondeterministic.\n<br /><br />\nA search <b>problem</b> is formally defined as having the following:\n<ul><li><b>State space</b>: the set of all possible <b>states</b> of the environment.</li>\n<li><b>Initial state</b>: the initial state that the agent is initially in</li>\n<li><b>Goal state(s)</b>: the target state that the agent attempts to reach, can be multiple. A method e.g. <i>Is-Goal(s)</i> should return boolean indicating if reached.</li>\n<li><b>Action(<i>s</i>)</b>: a method which returns a finite set of actions that can be executed given a state <i>s</i></li>\n<li><b>Transition model</b>: a method which denotes the result of each action, e.g. <i>Results(s,a)</i> returns the result of applying action <i>a</i> to state <i>s</i>.</li>\n<li><b>Action cost function</b> <i>c(s,a,s')</i> which denotes the numeric cost of applying an action <i>a</i> on state <i>s</i> to achieve state <i>s'</i>.\nThis cost function should reflect the agent's own performance measure.</li></ul>\nAny sequence of actions leads forms a <b>path</b>, a path that leads to the goal state is a <b>solution</b> and an <b>optimal solution</b> is one that minimizes the path cost.\nWe can also visualize the solution in a <b>graph</b> in which edges are actions between the vertices which are states.\n<br /><br />\nOne final thing to consider is that these representations are a <b>model</b> and these have a specific <b>level of abstraction</b> that removes as much information as possible\nwhile ensuring that <i>validity</i> is preserved and that abstract solutions can be carried out for the agent. The process of removing these other details is <b>abstraction</b>.\n\n## AIMA Chapter 3.2 Example Problems\n It is important first to distinguish between a <b>standardized problem</b> and actual <b>real-world problems</b>. Some examples of this include:\n<ul><li><b>State space</b>: the set of all possible cells. For <b>vacuum world</b>, if there are <i>n</i> cells there are <i>n * 2<sup>n</sup></i> states. This can also be a\n(partially filled) Sudoku board, a description of all the tiles in the <b>sliding-tile puzzle</b>, a <b>sobokan board</b>, the real numbers in Knuth's conjecture, or the\nlocation (e.g. an airport) and current time in the <b>route-finding problem</b>. The Sobokan example can have <i>n * n!/(b!(n-b)!)</i> or over 200 trillion possibilities\nfor an 8x8 board, Knuth's conjecture has infinite possible states.</li>\n<li><b>Initial state</b>: Any state in <b>grid world</b> or any arrangement of <b>sliding-tiles</b> (note that a goal can be reached by about half of the states),\nthe number 4 in Knuth's conjecture, or the user's home airport.</li>\n<li><b>Goal state(s)</b>: For <b>vacuum world</b>, the state in which every cell is clean. For <b>sliding-puzzle</b>, any state, but typically that in which the numbers are in\norder, for Knuth's conjecture, the desired positive integer, and for <b>route planning</b> a specified destination city.</li>\n<li><b>Action(<i>s</i>)</b>: These can be <b>absolute</b> movement actions like <i>Upward, Downward</i> or <b>egocentric actions</b> with respect to the agent, like\n<i>Forward</i>, <i>Backward</i>, etc. for the vacuum world. For the <b>sliding-tile puzzle</b>, this involves moving a tile in one of the cardinal directions to an empty\nspace. For Knuth's conjecture, this can be either the square root, floor, or factorial operations. And for the <b>route-finding</b> problem this is taking any flight\nfrom the current location that leaves after the current time and leaves enough time for a within-airport transfer.</li>\n<li><b>Transition model</b>: certain actions like <i>Suck</i> remove dirt from one cell, resulting in a new state, and motion activities leave the vacuum in a different\nposition. The puzzle pieces move into the empty space if possible, creating a new empty space. The transition model for Knuth's conjecture is the mathematicaloperations,\nand for the route planning problem, it will be the new destination city and flight's arrival time.</li>\n<li><b>Action cost function</b> Each action costs 1 for the <b>vacuum world</b>, <b>sliding-tile puzzle</b> and Knuth's conjecture problems. For the <b>route planning</b>\nalgorithm, this can be a combination of cost (monetary), time (either waiting and/or flying), and other factors like seat quality, flight points, etc.</li></ul>\nSome other more complicated problems that can be solved include the <b>traveling salesperson problem (TSP)</b> which is a type of <b>touring problem</b> in which every city on\na map is to be visited once and <b>VLSI layout</b> which groups cells into an ideal <b>cell layout</b> and then designing a <b>channel routing</b> procedure.\n\n\n## AIMA Chapter 3.3 Search Algorithms\nA <b>search algorithm</b> takes as input a search problem and returns either a solution or failure. A <b>search tree</b> is a representation fo the paths\ntaken from the initial state over the state-space graph that reach towards a goal state. Each <b>node</b> in the search tree corresponds to a state in the search tree and\nits edges correspond to actions from that node. From the root node, we <b>expand</b> each node by considering the available ACTION and RESULT functions to see what other\n<b>child node</b> or <b>successor node</b> can be generated from the current node - now the <b>parent node</b>. The nodes in the search tree which have been reached already\nare the <b>frontier</b> and as a node gets added to this it is considered <b>reached</b>. The frontier hence <b>separates</b> two regions of the state-space graph: an\ninterior where every node has been expanded, and the exterior which has not been reached.\n<br /><br />\nOne such search algorithm is <b>best-first search</b> in which the node <i>n</i> that is chosen has the minimum value of an <b>evaluation fucntion</b> <i>f(n)</i>. The\nalgorithm proceeds on each iteration by checking if it is the goal state (and returning if so), or expanding on its neighbors otherwise. Each node is added to the frontier\nif it has not been reached before or if it has a lower path cost than previously. The algorithm returns either a path to the goal or indicates failure.\n<br /><br />\nA <b>node</b> in a tree is represented by a data structure which has four components:\n<ul><li><i>node</i>.STATE: the state the node corresponds with</li>\n<li><i>node</i>.PARENT: the node from which the node was generated</li>\n<li><i>node</i>.ACTION: the action from which the node was reached (from the parent node)</li>\n<li><i>node</i>.PATH-COST: the total cost along path from initial state to the node, synonymous to <i>g(node)</i></li></ul>\nFollowing the PARENT pointers back allow one to recover the entire states and actions leading to a node. Doing this from the goal leads to the solution.\n<br /><br />\nThe <b>frontier</b> needs to support operations to check if it IS-EMPTY, and to POP and ADD elements. Some types of <b>queues</b> that can be used include:\n<ul><li>A <b>priority queue</b> pops the element with minimum cost according to a function <i>f</i></li>\n<li>A <b>FIFO queue</b> pops the node that was added first</li>\n<li>A <b>LIFO queue</b> (or a <b>stack</b>) pops the node that was added most recently</li></ul>\nEvaluate performance on:\n<ul><li><b>Completeness</b>: guaranteed to find solution when there is one, and correctly report failure if not?</li>\n<li><b>Cost optimality</b>: guaranteed to return lowest cost solution?</li>\n<li><b>Time complexity</b>: how long does it take? measured by # of states/actions considered.</li>\n<li><b>Space complexity</b>: how much memory needed to search?</li></ul>\nSearch is <b>graph search</b> if it checks for <b>redundant path</b>, otherwise <b>tree-like search</b>.\n\n\n## AIMA Chapter 3.4 Uninformed Search Strategies\nTypes of uninformed search include:\n<ol><li><b>Breadth-First Search</b>: a strategy in which a root node is expanded, then its successors, and their successors. Complete even on an infinite state space. \nOne optimization is to do an <b>early goal test</b>, i.e. checking if a goal state has been reached as soon as it has been <i>generated</i> \nas opposed to when dequeued as in with a <b>late goal test</b>.</li>\n<li>Dijkstra's algorithm or <b>uniform cost search</b> spreads out in waves of uniform path-cost. It can be implemented as a call to BEST-FIRST-SEARCH with PATH-COST\nas the evaluation function. BFS tests for the goal state only when it expands a node, not when it generates it.</li>\n<li><b>Depth first search</b> is tree-like search, expands node until it cannot anymore, then reverts to state where there is more to explore. In\n<b>backtracking search</b>, even less memory used as it generates one successor at a time, modifying state description instead of allocating memroy for a \nbrand-new state.</li>\n<li><b>Depth-limited search</b> is a version of DFS where we treat anything beyond a depth-limit <i>l</i> and treat nodes at that depth at terminal. A good heuristic\nis the <b>diameter</b> of state-space graph, but this is not always known. <b>Iterative deepening search</b> works by doing depth-limited search for 1, 2, etc. until\na cutoff value is reached.</li>\n<li><b>Bidirectional search</b> expands from the initial state and backwards from the goal state(s). Need to\nreverse the problem and order of visited solutions for backward direction.</i>\n</ol>\n\n<table>\n<tr><td>Criterion</td><td>Breadth-First</td><td>Uniform-Cost</td><td>Depth-First</td><td>Depth-Limited</td><td>Iterative Deepening</td><td>Bidirectional</td></tr>\n<tr><td>Complete?</td><td>Yes<sup>1</sup></td><td>Yes<sup>1,2</sup></td><td>No</td><td>No</td><td>Yes<sup>1</sup></td><td>Yes<sup>1,4</sup></td></tr>\n<tr><td>Optimal cost?</td><td>Yes<sup>3</sup></td><td>Yes</td><td>No</td><td>No</td><td>Yes<sup>3</sup></td><td>Yes<sup>3,4</sup></td></tr>\n<tr><td>Time</td><td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>1+floor(C*/e)</sup>)</i></td><td><i>O(b<sup>m</sup>)</i></td><td><i>O(b<sup>l</sup>)</i></td>\n<td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>d/2</sup>)</i></td></tr>\n<tr><td>Space</td><td><i>O(b<sup>d</sup>)</i></td><td><i>O(b<sup>1+floor(C*/e)</sup>)</i></td><td><i>O(bm)</i></td></td><td><i>O(bl)</i></td></td><td><i>O(bd)</i></td>\n<td><i>O(b<sup>d/2</sup>)</i></td></tr>\n</table>\nwhere <i>b</i> is the branching factor, <i>m</i> is the maximum depth of the search tree, <i>d</i> is the depth of the shallowest solution or <i>m</i> when no solution,\nand <i>l</i> is the depth-limit.<br />\n<sup>1</sup>: complete if <i>b</i> is finite and state space is either finite or has a solution<br />\n<sup>2</sup>: complete if all actions have cost >= <i>e</i> > 0<br />\n<sup>3</sup>: cost-optimal if actions are all identical<br />\n<sup>4</sup>: if both directions are breadth-first or uniform-cost\n\n## Module 3 Lecture Notes\n Your summary here \n\n## Module 3 Definitions\n* **Problem-solving agent** - an agent that can formulate a plan via search\n* **Atomic** - representations of the world that are considered as a whole with no visible internal structure\n* **Informed algorithm** - an algorithm in which an agent has information to estimate its distance to a goal state\n* **Uninformed algorithm** - an algorithm in which an agent has no information to estimate its distance to a goal state\n* **Solution** - a sequence of actions through which an agent can reach its goal state\n* **Open-loop system** - a system in which an agent can ignore the percepts once its solution is determined\n* **Closed-loop system** - a system in which the environment can change, requiring the agent to rely on percepts\n* **State space** - the set of possible states that the environment can be\n* **Initial state** - the initial state of the environment that the agent starts the problem in\n* **Goal states** - states that the agent intends to achieve\n* **Actions** - the choices an agent can make\n* **Transition model** - a formula that describes what an action does\n* **Action cost function** - a function that describes the cost an agent occurs in an action\n* **Path** - a sequence of actions\n* **Optimal solution** - the solution that has the lowest path cost to the agent\n* **Model** - an abstract mathematical representation\n* **Abstraction** - the process of removing details from a representation\n* **Level of abstraction** - the degree to which detail has been omitted for an action\n* **Standardized problem** - an illustrative problem to show an example of problem-solving\n* **Real-world problem** - a problem that arises from actual use case\n\n\n# Module 4: Informed Search\n## AIMA Chapter 3.5 Informed (Heuristic) Search Strategies\n<b>Informed search</b> strategies use a domain-specific <b>heuristic function</b> <i>h(n)</i> to find solutions more efficiently.\n<b>Greedy best-first search</b>: expands the node with the lowest <i>h(n)</i> value - finds a solution, but may ignore options that initially appear to be not optimal.\n<br /><br />\n<b>A* Search</b>: Best-first search using <i>f(n) = g(n) + h(n)</i> where <i>g(n)</i> is path cost from initial state to node <i>n</i>.\nA* relies on having an <b>admissible heuristic</b> and is cost-optimal with a <b>consistent</b> heuristic. However, can expand an exponential number of nodes.\n<br /><br />\nIf  a <b>satisficing</b> solution is okay, then it is possible to run A* with an \n<b>inadmissible heuristic</b> that might miss optimal solution but further reduces the number of nodes expanded. An example is the <b>detour index</b>. \nThis idea can be applied to any problem using the <b>weighted A* search</b>\nwhere the heuristic is weighted so evaluation function <i>f(n) = g(n) + W * h(n)</i> for some <i>W > 1</i>. If the optimal solution has cost <i>C*</i>, then\nweighted A* has a solution between <i>C*</i> and <i>W * C*</i>, generally closer to <i>C*</i>. Some types of suboptimal search include <b>bounded\nsuboptimal search</b>, <b>bounded-cost search</b>, <b>unbounded-cost search</b> (e.g. <b>speedy search</b>).\n<br /><br />\nMemory-Bounded Search:\n<ul><li><b>Beam Search</b>: limits size of the frontier, keeping only the <i>k</i> nodes with the best <i>f</i>-scores. Clearly incomplete and suboptimal, but can execute\nfast because expands fewer nodes, i.e. exploring only optimal portions of contour.</li>\n<li><b>Iterative-deepening A* Search</b>: uses cutoff of <i>f</i>-cost (<i>g + h</i>) - for each iteration, the cutoff value is the smallest <i>f</i>-cost that exceeds that of\nthe previous iteration. If optimal solution has cost <i>C*</i>, no more than <i>C*</i> iterations.</li>\n<li><b>Recursive best-first search</b>: mimics best-first search, but instead of continuing indefinitely down a path, uses the <i>f</i>-limit to keep track of\nthe <i>f</i>-value of the best alternative path from any ancestor of a current node. If the current node exceeds this <i>f</i>-limit, thenRBFSt unwinds back to alternative\npath. As recursion unwinds, RBFS replaces the <i>f</i>-value of cost along the <b>backed-up value</b> or best <i>f</i>-value of its children.</li>\n<li><b>MA*</b> (Memory-Bounded A*) and <b>SMA*</b> (Simplified MA*) are better when we can determine how much memory is available. SMA* expands\nA* until memory is full, then drops the leaf node with highest <i>f</i>-value, only regenerating this when all the other paths are worse than the\nforgotten path. Complete if any reachable solution is less than memory, optimal if any optimal solution is reachable, but can be very inefficient if forced to switch\nback and forth between many candidates and re-generate tree.</li></ul>\n<br />\nFor bidirectional Search, need to run in both directions.\n\n## AIMA Chapter 3.6 Heuristic Functions\nWhen considering different heuristic functions, important to consider <b>effective branching factor</b>. If the total number of nodes generated by A* is <i>N</i>\nand solution depth is <i>d</i> then <i>b*</i> is the branching factor that uniform tree of depth <i>d</i> would have to have in order to contain <i>N + 1</i> nodes. Thus,\n<i>N + 1 = 1 + b* + (b*)^2 + ... + (b*)^d</i>, e.g. a solution at depth 5 after 52 nodes has EBF of 1.92. A given heuristic <i>h</i> reduces the <b>effective depth</b> by a \nconstant <i>k<sub>h</sub></i> compared to true depth, reducing search cost from uninformed <i>O(b<sup>d</sup>)</i> to <i>O(b<sup>d-k<sub>h</sub></sup>)</i>. A value close to 1 is\nbetter, so for two heuristics <i>h<sub>1</sub></i> and <i>h<sub>2</sub></i>, it is said <i>h<sub>2</sub></i> <b>dominates</b> <i>h<sub>2</sub></i> if it expands fewer nodes as\nevery node expanded in the better function will be expanded in the other.\n<br /><br />\nThere are several ways to generate a heuristic:\n<ol><li>Create a solution for a <b>relaxed problem</b>, e.g. <b>Manhattan distance</b> solves the 8-block puzzle when the condition that squares cannot be moved to non-empty \nspots is lifted.</li>\n<li>Derive he solution cost of a <b>subproblem</b> of a problem, e.g. only considering tiles 1-4 in the 8-block puzzle. These states can be cached in a <b>pattern \ndatabase</b>, then using that subproblem cost as an admissible heuristic. The other subproblem - 5-8 may have some overlapping moves with this subproblem, \nso the sum of both may overestimate; then, a <b>disjoint pattern database</b> which excludes common moves should be used.</li>\n<li><b>Precomputation</b> of some optimal path costs. One approach is to pick 10 to 20 <b>landmark points</b> and then for each landmark <i>L</i> and other vertex\n<i>v</i>, compute and store <i>C*(v,L)</i>, the optimal path from <i>v</i> to <i>L</i>. This creates an efficient but inadmissible heuristic: <i>h<sub>L</sub>(n) =\nmin<sub>L in Landmarks</sub>C*(n,L) + C*(n,goal)</i> since this is either exact (the node goes through a landmark) or overestimates. \n<br />\nOne can also compute several <b>shortcuts</b>, e.g. from CA to NY. This is still not admissible, but the <b>differential heuristic</b><i>h<sub>L</sub>(n) =\nC*(n,L) - C*(goal,L)</i> modifies this slightly so the landmark is now somewhere past the goal. Landmarks should be spread out; they can also be picked based on frequency. \nThey work well in real-world situations because roads are often built along commonly travelled paths.</li>\n<li>Machine Learning algorithms on the <b>metalevel state space</b> - the entire search tree as opposed to just the current state - e.g. a map of is an <b>object-level\nstate space</b>. <b>Metalevel learning</b> algorithms can learn to avoid certain subtrees and thus minimize the <b>total cost</b> using techniques like reinforcement learning -\nthese work better when <b>features</b> relate to state's heuristic value instead of just describing it.</ol>\n\n\n## Module 4 Lecture Notes\n Your summary here \n\n## Module 4 Definitions\n* **Informed search** - seearch strategy that uses domain-specific hints to find solutions more efficiently than uninformed search\n* **Heuristic function** - <i>h(n)</i> is the <i>estimated</i> cost of  the shortest path from <i>n</i>to a goal-state.\n* **Admissibility** - an admissible heuristic is one that is optimistic, i.e. never overestimates the coast to reach a goal\n* **Consistency** - a heuristic is consistent if for every node <i>n</i>, its successors <i>n'</i> satisfy <i>h(n) <= c(n,a,n') + h(n')</i>\n* **Triangle inequality** - one side of a triangle cannot be larger than the other two sides combined\n* **Contours** - an outline of points equidistant from ainitial state\n* **Monotonic** - the path cost is always increasing along a certain path\n* **Surely expanded nodes** - all nodes reachable from an initial state on a path where every node has less than optimal cost\n* **Optimally efficient** - an algorithm such that any algorithm that achieves the same state must pass through the same nodes\n* **Prunes** - eliminating possible candidates for expansion without having to first examine them\n* **Satisficing** - good enough\n* **Inadmissible heuristic** - a heuristic that overestimates\n* **Detour index** - heuristic which applies a multiplier to the straight-line distance (e.g. 10 mi apart -> estimate 13 mi)\n* **Bounded suboptimal search** - a solution that is guaranteed to be within a constant factor <i>W</i> of the optimal cost\n\n\n# Module 5: Adversarial Search and Games\n## AIMA Chapter 5.1 Game Theory\nIn game theory, there are 3 approaches: \n<ol><li>The behaviors of a large number of agents can be viewed in aggregate as an <b>economy</b>.</li>\n<li>Adversarial agents are part of the environment, which is nondeterministic.</li>\n<li>Explicitly model the adversarial agents with techniques like adversarial game-tree search.</li></ol>\nHere we focus on the 3rd and use an algorithm to define the optimal move via minimax search, which is an optimization of <i>AND-OR</i> search.\nFurther techniques like <b>pruning</b> which leads the algorithm to ignore portions of the search tree that will not result in an optimal move.\nIn order to determine which method is winning, we apply a heuristic <b>evaluation function</b> to determine which agent is winning based on the current state of the board.\n<br />\nMany of the games studied within AI are deterministic, two-player, turn-taking, <b>perfect information</b> (fully observable), <b>zero-sum games</b> (what is good for one\nplayer is bad for the other). In this context, <b>move</b> is synonymous with \"action\" and <b>position</b> is synonymous with \"state\". Here, the two players are called \n<i>MAX</i> and <i>MIN</i>, where <i>MAX</i> moves first, and the players take turns until the game reaches a terminal state. At the end, the winning party gets the points \nand the other party is given a penalty. The game can be formally defined with the following elements:\n<ul><li>The <b>initial state</b> is the state of the game when it is first initialized</li>\n<li><i>TO-MOVE(s)</i>: Returns which agent's turn it is to move at state <i>s</i>.</li>\n<li><i>ACTION(s)</i>: Returns the set of all legal moves in <i>s</i></li>\n<li><i>RESULT(s, a)</i>: The <b>transition model</b>, which is the state of the game after action <i>a</i> is taken in state <i>s</i>.</li>\n<li><i>IS-TERMINAL(s)</i>: Returns the result of the <b>terminal test</b>, which is true when the game is over (and otherwise false). States in which the game passes the\nterminal test are called <b>terminal states</b>.</li>\n<li><i>UTILITY(s, p)</i>: Returns the result of the <b>utility function</b> (otherwise known as the objective or payoff function), which assigns a numeric value to player\n<i>p</i> for the game ending in the terminal state <i>s</i> For chess, a win is defined as 1 point, a loss as 0 points, and a draw as 1/2 point. In other games like\nbackgammon the outcomes can be in the range of 0 to 192.</li></ul>\nThe initial state, the <i>ACTION</i>, and <i>RESULT</i> functions define the <b>state space graph</b>, which is a graph where the edges are moves, the vertices are states, and\na state can be reached by multiple moves. A <b>search tree</b> is superimposed over part of this graph to determine what is the best move to make. A <b>game tree</b> is a search \ntree which follows every sequence of moves leading up to a terminal state. The game tree can be infinite if the game allows for infinitely many repeating positions or if \nthe state space graph is unbounded.\n\n## AIMA Chapter 5.2 Optimal Decisions in Games\nA modified version of <i>AND-OR</i> search called  <b>minimax search</b> models this behavior. Here, a turn taken by one player is a <b>ply</b>, and at each\nlayer the goal is the <b>minimax value</b> for each state of the tree, denoted by <i>MINIMAX(s)</i>. For a terminal state, this is its utility. <i>MAX</i> prefers\na move to a state of maximum value while <i>MIN</i> prefers a move to a state of minimum value. This results in the formula:\n<table>\n<tr><td><i>MINIMAX(s)</i>=</td><td></td></tr>\n<tr><td><i>UTILITY(s,MAX)</i></td><td>if <i>IS-TERMINAL(s)</i></td></tr>\n<tr><td><i>max<sub>a in Action(s)</sub> MINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n<tr><td><i>max<sub>a in Action(s)</sub> MINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n</table>\n<br />\ne.g. a first <i>MIN</i> node has values 3, 12, and 8, so the minimax value is 3. The other successor states of the root have minimax values of 2,\nso the root has a minimax value of 3. The root is the <b>minimax decision</b> or optimal choice for <i>MAX</i> because it leads to the state with highest minimax value. The\nminimax algorithm works by computing the best move for <i>MAX</i> by choosing the state with the highest <i>MINIMAX</i> value. It recursively proceeds all the way down to\nthe leaves and then <b>backs up</b> the minimax values as the recursion unwinds. The time complexity is <i>O(b<sup>m</sup>)</i> and the space complexity is <i>O(bm)</i> when all\nactions are generated at once or <i>O(m)</i> when generated one at a time.\n<br /><br />\nIn multiplayer games, players form <b>alliances</b>, formally or informally, e.g. players A and B target C if C appears weaker.\n<br /><br />\nTo reduce the number of game states, <b>pruning</b> large parts of the tree that make no difference to the outcome can be done as in <b>alpha-beta pruning</b>. This involves\npassing additional parameters to <i>MAX_VALUE(state, alpha, beta)</i> where\n<ul><li><i>alpha</i> = the value of the best (i.e. highest-value) choice along the path for <i>MAX</i> (think: \"at least\")</li>\n<li><i>beta</i> = the value of the best (i.e. lwoest-value) choice along the path for <i>MIN</i> (think: \"at most\")</li></ul>\nThe branch factor can become <i>b<sup>0.5</sup></i> instead of <i>b</i>, so we only need to examine <i>O(b<sup>m/2</sup>)</i> instead of <i>O(b<sup>m</sup>)</i>\"\nnodes. Some other considerations include <b>iterative deepening</b> to expand a level before moving on to next, focusing on <b>killer moves</b> and try them first and to\nconsider using a <b>transposition table</b> that caches the heuristic value of states in the case of <b>transpositions</b> or permutations that result in the same game state.\n<b>Type A strategy</b> considers all possible moves to a certain depth and evaluates them using a heuristic while a <b>Type B Strategy</b> ignores moves that look bad,\ninstead focusing on a <i>deep but narrow</i> portion of the tree.\n\n## AIMA Chapter 5.3 Heuristic Alpha–Beta Tree Search\nBecause computation time is limited, we cut off search early and use a heuristic <b>evaluation function</b> on the states, effectively treating them as terminal.\nWe replace the <i>UTILITY</i> function with the <i>EVAL</i> function, and also replace the terminal test with a <b>cutoff test</b> - which returns <i>True</i> for terminal\nstates or states past the cutoff depth. This gives the formula:\n<table>\n<tr><td><i>H-MINIMAX(s)</i>=</td><td></td></tr>\n<tr><td><i>EVAL(s,MAX)</i></td><td>if <i>IS-CUTOFF(s)</i></td></tr>\n<tr><td><i>max<sub>a in Action(s)</sub> H-MINIMAX(RESULT(s,a),d+1)</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n<tr><td><i>max<sub>a in Action(s)</sub> H-MINIMAX(RESULT(s,a),d+1)</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n</table>\n<br />\nFor terminal states, <i>EVAL(s,p)=UTILITY(s,p)</i> and for nonterminal states, <i>UTILITY(loss,p)<=EVAL(s,p)<=UTILITY(win,p)</i>. Most evaluation functions work by taking on \nthe <b>expected value</b> various <b>features</b> of the multiple <i>categories</i> or <i>equivalence classes</i>  and the <i>proportion</i> of each states with each outcome.\nA simple example is taking on the <b>material value</b> of each chess piece and then taking a <b>weighted linear function</b>, e.g. <i>EVAL(s) = w<sub>1</sub>f<sub>1</sub>(s) +\n... + w<sub>n</sub>f<sub>n</sub>(s)</i>. Some further modifications include applying the evaluation function only to <b>quiescent</b> positions where there is no pending move\nwhich swings the evaluation, and doing an extra <b>quiescence search</b> otherwise. There is also the <b>horizon effect</b> in which certain moves can be pushed off until\nlater, which can be mitigated by <b>singular extensions</b>, moves that are clearly better than others.\n<br /><br />\nAnother issue is with alpha-beta pruning potentially <b>forward pruning</b> some states that appear poor but are better. Some alternatives include <b>beam search</b> which\nonly includes the top <i>n</i> candidates but is similarly problematic, or methods like <i>PROBCUT</i> which considers moves <i>probably</i> instead of <i>provably</i> outside\nthe <i>(alpha, beta)</i> window. Another technique called <b>late move reduction</b> assumes moves are selected well, and discards moves that are later on. More complicated\nalgorithms like <i>STOCKFISH</i> also exist.\n<br /><br />\nFor openings and closings, a program can be trained to use a <i>table lookup</i> instead of calculating specific states. These include common chess openings. The table\ncan also be constructed by <b>retrogrdae</b> minimax search, which works backwards to identify particularly effective states.\n\n\n## AIMA Chapter 5.5 Stochastic Games\n<b>Stochastic games</b> introduce an element of chance by introducing random elements, e.g. throwing a dice or drawing a card. An example of this is Backgammon.\nBecause Black cannot know with certainty what outcome White will draw, the standard game tree from chess/tic-tac-toe cannot be used, but instead a game tree for Backgammon must\ninclude <b>chance nodes</b> in addition to the <i>MAX</i> and <i>MIN</i> nodes. The branches leading from each chance node represent the probability for each dice roll: while\nthere are 36 possible ways to roll two dice, each with 1/36 probability, order does not matter, i.e. a 4-2 is the same a s a 2-4. Therefore, there are only six distinct rolls.\nThe six doubles (e.g. 1-1,2-2,...6-6) each have probability of 1/36, e.g. P(3-3)=1/36, while the other 15 outcomes have a probability of 1/18.\n<br /><br />Now, since the goal is still to make the correct decision but the positions do not have definite values, we instead calculate the <b>expected value</b> of a\nposition, which is the average of all the possible outcomes over the chance nodes. This leads to the <b>expectiminimax value</b> for stochastic games, which is a generalization\nof the minimax value used with deterministic nodes. Terminal nodes for <i>MAX</i> and <i>MIN</i> work the same way as before. For chance nodes, the expected value which is the\nsum of the value over all outcomes, weighted by the likelihood or probability of each action:\n<table>\n<tr><td><i>EXPECTIMAX(s)</i>=</td><td></td></tr>\n<tr><td><i>UTILITY(s,MAX)</i></td><td>if <i>IS-TERMINAL(s)</i></td></tr>\n<tr><td><i>max(a) EXPECTIMINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MAX</i></td></tr>\n<tr><td><i>min(a) EXPECTIMINIMAX(RESULT(s,a))</i></td><td>if <i>TO-MOVE(s)=MIN</i></td></tr>\n<tr><td><i>sum(r) P(r) * EXPECTIMINIMAX(RESULT(s,r))</i></td><td> if <i>TO-MOVE(s)=CHANCE</i></td></tr>\n</table>\nHere, <i>r</i> represents a possible dice roll (or other random event) and <i>RESULT(s, r)</i> is the state of <i>s</i> with the additional fact that the dice roll has\nresulted in <i>r</i>.<br /><br />Now, with approximate values, the evaluation functions do not just need to give higher values to better positions, but they need to be more\ncareful - the program behaves differently if a change is made to some of evaluation values even if preference remains the same, e.g. [2,3,4,5] vs [20,300,400,5000]. Hence,\ninstead, the evaluation function returns values that are a positive linear transformation of the expected utility - for games with outcomes of win/lose this is the \n<b>probability</b>  of winning.<br /><br />Now, if minimax can run in <i>O(b<sup>m</sup>)</i> time, where <i>b</i> is the branching factor and <i>m</i>is the maximum depth\nof the tree, then expectimax must always take into account the number <i>n</i> of distinct roles, resulting in a runtime of <i>O(b<sup>m</sup>n<sup>m</sup>)</i>. This makes it\nvery impractical to run more than a few ply of a game like Backgammon.\n\n## AIMA Chapter 16.1 Combining Beliefs and Desires under Uncertainty\nGiven a state <i>s</i>, we can say that the state has a probability associated with it that can be expressed as <i>P(s)</i>. Then for each action <i>a</i> that\nan agent can tak, the resulting state and the uncertainty of it can be expressed by <i>P(s'|s,a)</i>, i.e. the probability that the action <i>a</i> in state <i>s</i> leads to\nstate <i>s'</i>. Alternatively, because the focus is the agent reaching state <i>s'</i>, the notation <i>P(RESULT(a)=s')</i> can be used. This is related to all the current\nstates by the formula \n<br /><i>P(RESULT(a)=s') = sum<sub>s</sub>P(s)P(s'|s,a)</i><br />\nDecision theory concerns itself with finding the choice of action that has the best <i>immediate</i> result. In order to calculate this, the agent uses the <b>utility\nfunction</b> <i>U(s)</i> across all the potential states resulting from an action to calculate the <b>expected utility</b> <i>EU(a)</i>, which is also equivalent to\nthe average utility value of the outcomes weighted by the probability of each outcome occuring, i.e.\n<br /><i>EU(a) = sum<sub>s'</sub>P(RESULT(a)=s')U(s')</i><br />\nThe general principle that is followed is that of <b>maximum expected utility (MEU)</b> which states that a rational agent should seek to choose the action that provides\nmaximum utility, i.e.\n<br /><i>action = argmax<sub>a</sub>EU(a)</i><br />\nThis principle <i>formalizes</i> the general notion that a rational agent should do the optimal thing but does not <i>operationalzie</i> the action that does. Obtaining\na good utility function and perceiving the world is still a challenge and the agent must also be able to make the right actions based on the values it obtains from this.\nThis results in the somewhat tautological view that the agent that acts to maximize its utility function should correctly reflect its performance measure, resulting in\nthe agent obtaining the highest possible score when weighted against all of the potential environments. This correlates the external performance with its internal\nrepresentation.\n\n## AIMA Chapter 16.2 The Basis of Utility Theory\n\nWe can express an agent's preferences with the following notation:\n<table>\n<tr><td><i>A > B</i></td><td>the agent prefers <i>A</i> instead of <i>B</i></td></tr>\n<tr><td><i>A ~ B</i></td><td>the agent is indifferent between <i>A</i> and <i>B</i></td></tr>\n</table>\nFor a given action, can consider the set of possible outcomes each with respective probability of occuring a <b>lottery</b> <i>L</i> such that:\n<br /><i>L = [p<sub>1</sub>,S<sub>1</sub>;p<sub>2</sub>,S<sub>2</sub>;...;p<sub>n</sub>,S<sub>n</sub>]</i>\nEach outcome <i>S<sub>i</sub></i> can either be an atomic sentence or another lottery. There are six main axioms of utility theory:\n<ol><li><b>Orderability</b>: Exactly one of <i>A > B</i>, <i>B > A</i> or <i>A ~ B </i> holds.</li>\n<li><b>Transitivity</b>: <i>(A > B) /\\ (B > C) -> (A > C)</i></li>\n<li><b>Continuity</b>: If a lottery <i>B</i> is in between <i>A</i> and <i>C</i> in preference, then there is some probability <i>p</i> for which agent will be indifferent\nbetween getting <i>B</i> in a lottery that yields <i>A</i> with probability <i>p</i> and <i>C</i> with probability <i>1-p</i>:\n<br /><i>A > B > C -> exists(p) [p;A;1-p,C] ~ B</i></li>\n<li><b>Substitutability</b>: Given two lotteries, w.l.o.g. <i>A</i> and <i>B</i>, then the agent is indifferent between two or more complex\nlotteries such that the only difference is that <i>B</i> is substituted for <i>A</i> in one of them, i.e.\n<br /><i>A ~ b -> [p,A;1-p,C] ~ [p,B;1-p,C]</i></li>\n<li><b>Monotonicity</b>: Given two lotteries with the same two possible outcomes, w.l.o.g. <i>A</i> and <i>B</i>, if an agent prefers <i>A</i> to <i>B</i>,\nthe agent also prefers the lottery that has a higher probability of <i>A</i> (and vice versa):\n<br /><i>A > B -> (p > Q <-> [p,A;1-p,B] > [q,A;1-q,B])</i></li>\n<li><b>Decomposability</b>:Compound lotteries can be reduced to simpler ones using various laws of probability; for example, two consecutive lotteries can be condensed into\na single lottery as follows:<br /><i>[p,A;1-p,[q,B;1-q,C]] ~ [p,A;(1-p)q,B;(1-p)(1-q),C]</i></li>\n</ol>\nThese axioms of utility theory are:\n<ul>\n<li><b>Existence of utility Function</b>: if an agent has preferences that obey the axioms of utility, it follows that there exists a function <i>U</i> such that\n<i>(U(A) > U(B)) <-> (A > B)</i> and <i>(U(A) = U(B)) <-> (A ~ B)</i></li>\n<li><b>Expected Utility of a Lottery</b>: the sum of the probability of each outcome multipled by the utility of each outcome, i.e.\n<br /><i>U([p<sub>1</sub>,S<sub>1</sub>;...;p<sub>n</sub>,S<sub>n</sub>] = sum<sub>i</sub>p<sub>i</sub>U(S<sub>i</sub>)</i></li>\n</ul>\nThese theorems state that a utility function <i>exists</i> but not necessarily that an <i>unique</i> one exists. It is easy to see that any linear transformation, e.g.\n<i>U'(S) = aU(S) + b</i> is still valid. An agent only needs a preference ranking on these states not the actual values. This is called a <b>value function</b>\nor an <b>ordinal utility function</b>\n\n## AIMA Chapter 16.3 Utility Functions\nOne shortcoming of utility functions is they do not account for individual's preferences, which can lead to behavior that may initially appear irrational\nbut is substnatiated. These can be realized through a process called <b>preference elicitation</b>, whereby an agent is presented with different choices and then asked\nwhat their preferences are.\n<br /><br />\nIt is flawed to assume people have a <b>monotonic preference</b> for more money; given a gamble to wager 1 million for 2.5 by a coin flip, most people would not as the\n<b>expected money value</b> is not equal. For very rich or very poor people, they are more likely to gamble, indicating that the EMV of money is possibly logarithmic:\npeople at the curves are <b>risk-averse</b> while those at the fringes are <b>risk-seeking</b>.\n<br /><br />\nWe also do not perceive the world perfectly and are calculating the true utility. In reality, we are calculating an estimate of the utility function\n<i>EU<sub>est</sub>(a)</i> and in order for this to be <b>unbiased</b> the error must be 0, i.e. <i>E(EU<sub>est</sub>(a) - EU(a)) = 0</i>. We can often calculate the maximum\nof <i>k</i> estimates and then end up calculating an <b>order statistic</b> overestimate, for <i>X*</i> being the largest in a set {<i>X<sub>1</sub>,...X<sub>k</sub></i>},\n<br /><i>P</i>(max{<i>X<sub>1</sub>,...X<sub>k</sub></i>} <= <i>x</i>) = <i>F(x)<sup>k</sup></i>\n<br />and by its derivative,\n<br /><i>P(x) = kf(x)(F(x))<sup>k-1</sup></i>.\n<br /><br />\nThe <b>optimizer's curse</b> states that the estimated expected choice is likely to be too high. A study that shows a vaccine is 90% effective is not likely to be 90% effective on\nfuture victims for example, and most times something that is effective on 9/10 patients in fact performs worse than something else which is effective on 8000/10000\npatients. We can aim to mitigate this with a Bayesian approach that uses an explicit probability model <b>P</b>(<i>EU<sub>est</sub>|EU</i>) of the error in utility rates.\n<br /><br />\nSome phenomenon that can be observed include:\n<ul><li>the <b>certainty effect</b>: people are more likely to choose certain outcomes over ambiguous outcomes that may have a higher utility value (even in expectation).\nPeople can be described as having <b>ambiguity aversion</b></li>\n<li>the <b>framing effect</b>: people are more likely to respond positively to a large positive number than a small negative number, e.g. a 80% chance of living is seen\nas better than a 20% chance of dying even though those are the same.</li>\n<li>the <b>anchoring effect</b>: people judge based on relative not absolute utility. Pricing an expensive item like a 300 dollar bottle of wine or an overpriced medium box of\npopcorn drives people to get the 55 dollar bottle or the large popcorn.</li>\nInvestigation into phenomena like this from the perspective of <b>evolutionary psychology</b> indicates that the brain's inherent decision making systems may not view the\nworld through probabilities.\n\n## Module 5 Lecture Notes\n Your summary here \n\n## Module 5 Definitions\n* **Transition model** - the resulting state after an action is taken\n* **Utility function** - function for the numeric value assigned to a player when the game terminates\n* **Expected utility** - the average value of the utility function over all the potential outcomes\n* **Maximum expected utility** - the principle that a rational agent should choose actions that maximize expected utility\n\n\n# Module 6: Constraint Satisfaction Problems\n## AIMA Chapter 6.1 Defining Constraint Satisfaction Problems\n Solution to a CSP must be both <b>consistent</b> and <b>complete</b>.\n<br /><br />\nIn the Map Coloring problem, there are constraints that different territories cannot be the same color as their adjacent territories. We can come up wtih constraints in the\nform <i>C = {(SA, WA), SA =/= WA}</i> and from this reduce the map to each territory having its own color. One helpful tool is a <b>constraint graph</b> where nodes are the\nvariables and edges are possible solutions. Reducing constraints can significantly reduce the output space 87%.\n<br /><br />\nIn the Job Scheduling problem, certain operations must occur before others. These operations may take a specific amount of time. These result in <b>precedence constraints</b> \nthat say a task must occur a certain amount of time before another, e.g. <i>T<sub>1</sub></i> + <i>d<sub>1</sub></i> <= <i>T<sub>2</sub></i> indicates task <i>T<sub>1</sub></i> \ntakes <i>d<sub>1</sub></i> time to complete and must come before <i>T<sub>2</sub></i>. Another type of constraint are the <b>disjunctive constraint</b> which specify one or \nanother action must come before the other, e.g. <i>T<sub>1</sub></i> + <i>d<sub>1</sub></i> <= <i>T<sub>2</sub></i> <b>or</b> <i>T<sub>3</sub></i> + <i>d<sub>3</sub></i> <= \n<i>T<sub>4</sub></i>.\n<br /><br />\nThere are several types of constraints:\n<ul><li><b>unary constraint</b>: A constraint that restricts the value of a single variable.</li>\n<li><b>binary constraint</b>: A constraint that relates two variables. A problem that is made up solely of unary and binary constraints is a <b>binary CSP</b>.</li>\n<li><b>global constraint</b>: A constraint which applies across an arbitrary number of variables. A common global constraint is <i>Alldif</i> which specifies each element\nmust be different, e.g. the elements in a sudoku box. Another example are <b>cryptarithmetic</b> puzzles. These can often be modeled as a <b>constraint hypergraph</b>\nwhere special nodes indicate <i>n</i>-ary relationships. Alternatively, auxiliary variables can be introduced to reduce these to binary constraints and a <b>dual graph</b>\ncan be used instead.</li></ul>\nThe above examples involve <b>discrete, finite domains</b>. Other problems that can come up include <b>infinite domains</b> which require relationships to be expressed in the\nform of <b>linear constraints</b> on integer variables (as opposed to explicitly defined tuples). It has been shown the solution when including <b>nonlinear constraints</b> is\nundecidable. Another variation involves using <b>continuous domains</b>, e.g. scheduling problems with very specific time constraints. These form the basis of <b>linear\nprogramming</b> problems. A third variation involves specifying <b>preference constraints</b> which indicate which solutions might be preferred. These can be solved by using\noptimization search methods that can be either path-based or local and are called a <b>constraint optimization problem</b>. Linear programs are a type of COP.\n\n\n## AIMA Chapter 6.2 Constraint Propagation: Inference in CSPs\n<b>Constraint satisfaction</b> aims to ensure <b>local consistency</b> including:\n<ul>\n<li><b>Node consistency</b> ensures a node meets its unary constraints.</li>\n<li><b>Arc consistency</b> holds if every value in the domain satisfies its binary constraints. AC-3 enforces this by: \n<ul><li>Initially, all the arcs in a CSP are placed in a queue.</li>\n<li>AC-3 then pops off a queue and makes its two nodes arc-consistent.</li>\n<li>If any node changes, enque all its neighboring arcs.</li>\n<li>If no values for a node, AC-3 terminates.</li></ul>\nRuntime is <i>O(cd<sup>3</sup>)</i>.</li>\n<li><b>Path consistency</b> uses implicit constraints that can be inferred by looking at triples of variables on binary constraints, i.e. ensuring that for a two-variable\nassignment {<i>X<sub>i</sub> = a, X<sub>j</sub> = b</i>}, an assignment is consistent iff there is an assignment to <i>X<sub>m</sub></i> that satisfies both \n{<i>X<sub>i</sub>, X<sub>m</sub></i>} and {<i>X<sub>m</sub>, X<sub>j</sub></i>}.\n<li><b><i>k</i>-consistency</b> <-> for any consistent assignment to a set of <i>k - 1</i> variables, the <i>k</i>-th value can always be assigned a consistent value.\nThis is a stronger form of propagation: 1-consistency is node consistency, 2-consistency is arc concistency, in a binary graph 3-consistency is path-consistency. A CSP\nis further <b>strongly <i>k</i>-consistent</b> if it is <i>k</i>-consistent, (<i>k</i>-1)-consistent, ..., 1-consistent. This means that for a <i>n</i>> node problem,\nwe choose a consistent value for <i>X<sub>1</sub></i> and are guaranteed consistent solutions for <i>X<sub>2</sub>,...,X<sub>n</sub></i> by 2-consistency, 3-consistency, etc.\nTherefore the time is <i>O(n<sup>2</sup>d)</i> but establishing <i>n</i>-consistency is NP-complete.</li>\n<li><b>global constraint</b></li>\n</ul>\nAnother high-order constraint is the <b>resource constraint</b> or the <i>Atmost</i> constraint, which specifies a maximum total for a subset of nodes, e.g. for 4 people\n if we have that <i>Atmost(10,P<sub>1</sub>,P<sub>2</sub>,P<sub>3</sub>,P<sub>4</sub>)</i> then we know for a sample domain {2,3,5,6}\nwill not satisfy this since 5 + 6 > 10, so we can remove 5 and 6 from all domains involving those 4.\n<br /><br />\nFor large domains that cannot be easily enumerated, we use <b>bounds propagation</b> by keeping track of the range that a variable can be in, e.g.\n<i>D<sub>1</sub> = [0,165]</i> and adjust this value as new constraints appear. A CSP is <b.bounds-consistent</b> if for every <i>X</i> variable in the CSP, there is a variable\n<i>Y</i> that satisfies the constraint between <i>X</i> and <i>Y</i>.\n<br /><br />\n<b>Sudoku</b> is a puzzle that involves 81 tiles and 27 <i>Alldiff</i> <b>units</b>: one for each row, column, and 3x3 box. AC-3 works for the easiest of Sudoku puzzles, PC-2 \ncan solve harder puzzles but with 255,960+ path constraints, more complicated strategies like isolating \"naked triples\" and removing them from other arcs required.\n\n\n## AIMA Chapter 6.3 Backtracking Search for CSPs\nIf we finish CSP process with multiple values, then need to <b>search</b> for a solution. One possible way is to do depth-limited search. But for domain\nof size <i>d</i> with leaf nodes at depth <i>n</i>, branch factor would be <i>nd</i> because <i>d</i> values can be assigned to any of <i>n</i> levels, so there are\n<i>n!*d<sup>n</sup></i> leaves for <i>d<sup>n</sup></i> arrangements. However, CSPs are <b>commutative</b> and so backtracking by a series of heuristics can get back the\n<i>n!</i> factor.\n<br /><br />\nSome important heuristics to consider are the <b>minimmum-remining-values</b> (MRV) heuristic, aka the \"most constrained variable\" or \"fail first\" heuristic. This is the\nvariable that is most likely to fail, as it has the fewest options, so the search can terminate earlier if it is deteced immediately (if it has no legal values). This\nprunes the search tree by minimizing the number of nodes expanded. Another heuristic is the <b>degree heuristic</b> by selecting the variables that have constraints on the\nlargest number of other unassigned variables. This is not as good as MRV but can be used as a deal-breaker.\n<br /><br />\nAfter a variable has been selected, the algorithm decides which order to examine the values by using the <b>least-constraining-value</b> heuristic, or picking the value that\nrules out the fewest choices for neighbors in the constraint graph, or fail-last. The dichotomy between the two is fail-first minimizes the number of nodes to expand but\nfail-last then maximizes the number of possible outcomes that can be carried out forward.\n<br /><br />\nRunning inference during the process of search can yield new domain restrictions. One is <b>forward checking</b>, whereby after a value is assigned,\nwe check the arc consistency for it. This is often more effective when combined with the MRV heuristic, making it an efficient way to incrementally compute the information\nthat will be used by MRV. It does not detect all the possibilities however, so stronger algorithms like <b>Maintaining Arc Consistency</b> (MAC) are useful this. It calls\nAC-3 again but without the entire queue just the values that are unassigned neighbors of <i>X<sub>i</sub></i>, then doing constraint propagation. If any variable has its domain\nreduced to empty, this backtracks immediately. This recursive propogation makes it more powerful than just forward checking alone.\n<br /><br />\nWhen a branch fails, a process of <b>chronological backtracking</b> to the <i>most recent</i> decision is done. Some better psosibilities are to instead look at the\n<b>conflict set</b> that causes the CSP to fail, and then <b>backjump</b> until an assignment in that set fails, and then try out the other possible values for this assignment.\nIf none are found, then the CSP will fail. However, every branch trimmed by backjumping would also be checked by forward pruning, so only one is needed.\n<br /><br />\n<b>Constraint learning</b> can further  avoid <b>no-good</b> assignments.\n\n\n## AIMA Chapter 6.4 Local Search for CSPs\nLocal search algorithms perform significantly better with CSPs. In one example, for the 8-queens problem, instead of starting with a valid placement for the\nleftmost assignment, which will typically create several assignments, we can instead choose a randomly conflicted variable, and change this to a value that will bring it\ncloser to the solution using the <b>min-conflicts</b> heuristic, which selects the value that results in the minimum conflicts with other variables. This is surprisingly\neffective at many CSPs, excluding the initial placement of queens, its runtime is more or less <i>independent of problem size</i> - it can solve the <i>million</i>-queens\nproblem in about 50 steps after the initial assignment. Research has shown this is because the solutions are densely distributed through the state space. Still, local\nsearch has shown potential for planning the Hubble Space Telescope observations and harder problems.\n<br /><br />\nOne challenge is that there could be millions of variable assignments that are only one conflict away from a solution. Plateau search, or methods that would allow one to\nmove to another state with the same exact score could help a local search off its plateau. This can be directed via methods like <b>tabu search</b>, which keeps a small\nlist of recently visited states and forbids revisiting those states. To escape from plateaus, simulated annealing can also be used.\n<br /><br />\nA final technique called <b>constraint weighting</b> places more weight on the most important constraints. All of the constraints are initially given a weight of 1, but\nafter each step in the local search algorithm, the algorithm will choose a variable and value pair to change and that will as a result have the lowest total weight of all\nthe violated constraints considered so far. Weights would then be adjusted by incrementing each constraint violated by the current assignment. This has two benefits:\nfirst, it adds topography to plateaus, making it possible to improve from the current state, and it adds learning: over time the higher priority constraints get assigned\nhigher values.\n\n## AIMA Chapter 6.5 The Structure of Problems\nFinding <b>independent subproblems</b> can have a significant impact on being able to find the optimal solution. Breaking a graph up into <b>connnected\ncomponents</b> significantly reduces the runtime - given <i>c</i> out of <i>n</i> variables where there are <i>n/c</i> subproblems and <i>d</i> amount of work to solve \noverall total, or <i>d<sup>c</sup></i> per component, the total work is <i>O(d<sup>c</sup>n/c)</i>, which is <i>linear</i> in\n<i>n</i>. Without the decomposition it is <i>O(d<sup>c</sup>)</i>, which is <i>exponential</i> in <i>n</i>.\n<br /><br />\nWhile completely independent subproblems are rare, there are other structures that are helpful, e.g. a <b>tree</b> can be solved in time linear\nin the number of variables. This is because of <b>directional arc consistency</b> or DAC. A CSP is consistent for variables if and only if every \n<i>X<sub>i</sub></i> is arc consistent with <i>X<sub>j</sub></i> for <i>j > i</i>. In order to do this, one picks a variable to be a root\nof tree and then orders the tree such that each variable appears after its parent in a tree, i.e. a <b>topological sort</b>. A tree with <i>n</i> nodes has <i>n - 1</i> edges\nand thus the graph can be made arc-consistent in <i>O(n)</i> steps, each of which will consider <i>d</i> possible domains for a runtime of <i>O(nd<sup>2</sup>)</i>.\n<br /><br />\n<b>Cutset conditioning</b> is a technique of assigning values to some variables so the remainder form a tree. If a cycle cutset has size <i>c</i>, then the total\nruntime of the following is <i>O(d<sup>c</sup>*(n-c)d<sup>2</sup>)</i>:\n<ol><li>Choose a subset <i>S</i> of the CSP's variables such that the residual constraint graph is a tree. then<i>S</i> is called a <b>cycle cutset</b>.</li>\n<li>For each possible assignment to the variables in <i>S</i> which satisfy all the constraints on <i>S</i>,\n<ol><li>remove from the domains of the remaining variables any values that are inconsistent with the assignemnt for <i>S</i>, and</li>\n<li>if the remaining CSP has a solution, return it together with the assignment for <i>S</i>.</li></ol></li></ol>\nA <b>tree decomposition</b> of the constraint graph is a transformation of it into a tree where each node contains a set of variables. It must follow three requirements:\n<ol><li>Every variable in the problem appears in at least one of the tree nodes.</li>\n<li>If two variables are connected by a constraint in the original problem, they must appear together (along with the constraint) in at least one of the tree nodes.</li>\n<li>If a variable appears in two nodes in the tree, it must appear in every node along the path connecting those nodes.</li></ol>\nTree decomposition has a runtime of <i>O(nd<sup>w+1</sup>)</i>, so whenever the cycle subset <i>c</i> and tree width <i>w</i> are of size <i>w < c + 1</i>, tree\ndecomposition is better.\n<br /><br />\nA final method is <b>value symmetry</b> which breaks the symmetry in assignments using a <b>symmetry-breaking constraint</b>.\n\n\n## Module 6 Lecture Notes\n Your summary here \n\n## Module 6 Definitions\n* **Factored representation** - a representation of a problem via a set of variables, each with a value\n* **Constraint satisfaction problem (csp)** - a pproblem that is solved when each of its variables has a value and the original constraints are satisfied\n* **Relation** - a tuple relationship between a variable and the set of values it can take on\n* **Assignment** - a set of values given to a set of variables\n* **Consistent** - an assignment that does not violate any of the constraints\n* **Complete assignment** - an assignment in which each variable has a value\n* **Partial assignment** - an assignment that leaves some variables incomplete\n* **Partial solution** - a partial assignment that is consistent\n* **Constraint graph** - a visualization of a CSP where nodes correspond to variables and edges represent constraints\n* **Precedence constraint** - a constraint on when an individual task can occur or how long it can take\n* **Disjunctive constraint** - a constraint that allows for one of multiple possibilties (logical OR)\n* **Discrete domain** - a domain involving real numbers, e.g. map-coloring and scheduling with time limits\n* **Continuous domain** - a domain involving non-exact start and end times, like specialized scheduling tasks\n* **Finite domain** - a domain that has an upper limit to its size\n* **Infinite domain** - a domain that has no known upper limit, e.g. the set of integers or strings\n* **Linear constraint** - a constraint in which each variable is expressed in linear from with a linear relationship\n* **Linear programming** - a type of continuous-domain CSP where constraints must be linear equalities or inequalities\n* **Unary constraint** - a constraint on a single value or variable\n* **Binary constraint** - a constraint that relates two variables\n* **Binary csp** - a CSP involving only unary and binary constraints\n* **Global constraint** - a constraint involving an arbitrary number of variables\n* **Cryptarithmetic** - a CSP involving assigning values to letters based on linear equations\n* **Constraint hypergraph** - a constraight graph for higher-order csps, where n-ary constraints are represented as nodes\n* **Dual graph** - a graph in which one variable is in one graph and the constraints are in another graph\n* **Preference constraint** - a constraint that expresses a preference but not an absolute trestriction\n* **Constrained optimization problem** - csps involving preferences that can be solved using optimization search methods\n\n\n# Module 7: Logical Agents\n## AIMA Chapter 7.1 Knowledge-Based Agents\n A generic knowledge-based agent, when given a precept, adds it ot the knowledge base, then asks the knowledge base for the best action given this new\ninformation, and then updates the knowledge-based agent after the action has occured, i.e.:\n<table>\n<tr><td><b>function</b> <i>KB-AGENT(percept)</i> <b>returns</b> an <i>action</i><br />   <b>persistent</b>: <i>KB</i>, a knowledge base and <i>t</i>, a counter, initially 0,\nindicating time</td></tr>\n<tr><td><i>TELL(MAKE-PRECEPT-SENTENCE(precept,t))</i></td></tr>\n<tr><td><i>action</i> <- <i>ASK(KB, MAKE-ACTION-QUERY(t))</i></td></tr>\n<tr><td><i>TELL(KB,MAKE-ACTION-SENTENCE(action,t))</i></td></tr>\n<tr><td><i>t</i> <- <i>t</i> + 1</td></tr>\n<tr><td><b>return</b> <i>action</i></td></tr>\n</table>\nHere, <b>propositional logic</b> is used instead of the more expressive but computationally expensive <b>first-order logic</b>. The central component of any knowledge-based\nagent would be its <b>knowledge base</b>, hereafter KB. The KB consists of a series of <b>sentences</b> (which are related but not the same as sentence sin natural language).\nThese sentences are expressed in <b>knowledge representation language</b> to assert some truths about the environment they exist in - sentences given as true initially\nand taken for granted are called <b>axioms</b>. New sentences can be added via the <i>TELL</i> and <i>ASK</i> operations. These both involve <b>inference</b> or the process\nof deriving new sentences from existing sentences.\n<br /><br />\nThe detials of the representation langauge are hidden inside the functions that implement the interface between both the sensors and actuators on one end and the reasoning\nsystem and its core representation on the other:\n<ol><li><i>MAKE-PERCEPT-SEQUENCE</i> constructs a setence that asserts the given precept into the KB.</li>\n<li><i>MAKE-PERCEPT-SEQUENCE</i> constructs a sentence indicating the best action based on the current KB.</li>\n<li><i>MAKE-ACTION-SENTENCe</i> constructs a sentence that asserts the chosen action was taken.</li></ol>\nThe agent and its goals are defined at the <b>knowledge level</b>, at which the agent should be able to simply state what it knows and be provided with the best action to take.\nThe specifics of how these are calculated <b>implementation level</b>, e.g. via string manipulation or propagating signals down a neural network, should be abstracted away\nfrom the program at the knowledge level.\n<br /><br />\nThere was originally some debate between the <b>declarative</b> approach, in which general sentences or truths are fed to a model which can then predicate actions, and\nthe <b>procedural</b> approach, which involves explicitly encoding desired behaviors. But now we understand that the best performing agents will incorporate elements of both \nand that declarative knowledge can often be compiled into the underlying procedural code to improve performance.\n\n## AIMA Chapter 7.2 The Wumpus World\n The <b>wumpus world</b> is a cave with several different passageways. The agent's goal is to traverse the cave and find the gold without falling into a pit \nor being eaten alive by the wumpus, either of which will result in the agent's death. The agent also has a bow and arrow it can use once to attempt to eliminate the wumpus\nand several sensors which help it identify potential scenarios. The task environment can be defined precisely via the following PEAS description:\n<ul><li><b>Performance measure:</b> The agent is given +1000 points for climbing out of the cave with the gold, -1 point for each action taken, -10 points to use the arrow,\nand -1000 points for dying - either by being eaten by the wumpus or falling into the pit.</li>\n<li><b>Environment:</b> The default environment is a closed 4x4 grid, surrounded by walls. The gold and wumpus are placed randomly from a uniform distribution at any of the\nother squares than [1,1], where the agent is placed. Every other square can also be a pit, with probability 0.2.</li> \n<li><b>Actuators:</b>Tgere are several actions that the agent can make:\n<ul><li><i>Forward</i>: If an agent bumps into a wall, nothing happens.</li>\n<li><i>TurnLeft</i> and <i>TurnRight</i> by 90 degrees.</li>\n<li><i>Grab</i>: The agent picks up the gold, can only be used when the agent is in the same square as the gold.</li>\n<li><i>Shoot</i>: Fires an arrow in the direction the agent is facing. The arrow continues a straight line until it encounters a wall or the wumpus. If it hits the wumpus,\nit kills the wumpus, which lets out a <i>Scream</i>. The agent can now travel to that square without being eaten.</li>\n<li><i>Climb</i>: The agent can climb out of the cave, can only be used at square [1,1].</li></ul></li>\n<li><b>Sensors:</b> For each of the five sensors, the agent will perceive a:\n<ul><li><i>Stench</i> in squares that are directly but <i>not</i> diagonally adjacent to the wumpus.</li>\n<li><i>Breeze</i> in squares that are directly adjacent to a pit.</li>\n<li><i>Glitter</i> in a square where there is gold.</li>\n<li><i>Bump</i> if the agent walks into a wall.</li>\n<li><i>Scream</i> that can be heard anywhere in the cave when a wumpus dies.</li></ul>\n</li></ul>\nThe wumpus environment is clearly deterministic, discrete, single-agent, and static, since the wumpus does not move. It is also a sequential environment since rewards are\ncontingent on moves being taken and is partially observable, because only a few aspects, e.g. the agent's location, the wumpus' health, if the bow and arrow have been used,\nare known at any given time. As the agent goes through the board, it makes conclusions based on previous observations to speculate on the location of pits and the wumpus,\nfor example. Despite this, 21% of all possible environments are utterly unfair, e.g. the gold is located in a pit, which forces the agent to choose to go home empty-handed.\nThe conclusion is <i>guaranteed</i> to be correct if the provided information is correct.\n\n## AIMA Chapter 7.3 Logic\n Logic has certain fundamental characteristics regardless of its form. These include its <b>syntax</b> and <b>semantics</b>. Together, these define the\n<b>truth</b> of each statement in each <b>possible world</b> or <b>model</b>. We say that if a sentence <i>alpha</i> is true in a model <i>m</i> that <i>m</i> <b>satisfies</b>\n<i>alpha</i> or alternatively, that <i>m</i> <b>is a model of</b> <i>alpha</i>.\n<br /><br />\nThe process of <b>entailment</b> involves seeing if a sentence <i>follows logically</i> from another sentence. <i>alpha</i> entails <i>beta</i> if and only if for every\nsentence in which <i>alpha</i> is true, <i>beta</i> is also true, i.e. <i>M(alpha)</i> is a proper subset of <i>M(beta)</i>. This is similar to how if<i>y = 0</i> then \n<i>yz = 0</i>. In the Wumpus World example, if the agent does not sense a breeze in [1,1], then it follows that there is no pit in [1,2] or [2,1], so the KB entails the \nstatement <i>alpha<sub>1</sub></i> = \"There is no pit in [1,2]\" but not the statement <i>alpha<sub>2</sub></i> = \"There is no pit in [2,2]\" as this has not been yet shown. \nThis also illustrates the process of <b>logical inference</b> or applying the definition of entailment using newly found information in the KB to derive new conclusions. \nIn particular, this illustrates the <b>model checking</b> algorithm, which enumerates all the possibilities of a KB to see if <i>M(KB)</i> is a proper subset of <i>M(a)</i>.\n<br /><br />\nOne important consideration is that any model used should be both <b>sound</b> or <b>truth preserving</b> and also <b>complete</b>. This means that if the sentences in the\ninitial KB are true, any of the derived sentences maintain the truth properties of the other sentences in the KB and the inference algorithm proceeds by deriving all\nsentences that follow from the KB in each iteration. It follows from this that any derived sentences correspond to real-world relationships and imply the truthness of other\nproperties in the real world.\n<br /><br />\nAnother consideration is <b>grounding</b> - i.e. how we know that statements in the KB are true in the world. The soundness of these statements come from the fact that\nstatements added to the KB come from the process of sensing in the agent. While there is not a direct representation always for derived sentences, these come through\na process of <b>learning</b>, which can be fallible but one should still be optimistic given a good learning procedure.\n\n## AIMA Chapter 7.4 Propositional Logic: A Very Simple Logic\n<b>Propositional logic</b> involves representing <b>atomic sentences</b> by a single <b>propositional symbol</b>. From this, <b>complex sentences</b>\ncan be constructed by stringing together atomic statements with <b>logical connectives</b>. These are:\n<ul><li>~ (not): the <b>negation</b> returns the opposite truth value of the atomic sentence it is negating. A <b>literal</b> in this context can either be a\n<b>positive literal</b> (an atomic statement) or a <b>negative literal</b> (a negated atomic statement).</li>\n<li>/\\ (and): the <b>conjunction</b> has parts that are referred to as <b>conjuncts</b>. It is <i>true</i> if and only if all the conjuncts are true.</li>\n<li>\\/ (or): the <b>disjunction</b> has parts that are referred to as <b>disjuncts</b>. If either (or both) of the disjuncts are <i>true</i>, then so is the disjunction.\nThis is notably different from the colloquial \"or\" which means \"one or the other\" (in logic, this is the \"exclusive or\" or (xor) operator).</li>\n<li>-> (implies): the <b>implication</b> can be broken into two parts: the <b>antecedent</b> or <b>premise</b> and the <b>consequent</b> or <b>conclusion</b>. It is\n<i>true</i> when the premise and conclusion are both <i>true</i> or when the premise is <i>false</i>, but is <i>false</i> when the premise is <i>true</i> but the\nconclusion is <i>false</i>. When the premise is <i>false</i>, the implication is <i>vacuously true</i> as the implication makes no claim about the environment\nwhen the premise is <i>false</i>.</li>\n<li><-> (if and only if): the <b>biconditional</b> is only <i>true</i> if <i>both</i> statements are true.</li></ul>\nThe BNF (Backus-Naur) Form states the grammar of sentences in propositional logic, along with operator precedences.\n<i><table>\n<tr><td>Sentence</td><td>-> AtomicSentence | ComplexSentence</td></tr>\n<tr><td>AtomicSentence</td><td>-> True | False | P | Q | R | ...</td></tr>\n<tr><td>ComplexSentence</td><td>-> ( Sentence) )</td><tr>\n<tr><td></td><td>|  ~Sentence</td></tr>\n<tr><td></td><td>|  Sentence /\\ Sentence</td></tr>\n<tr><td></td><td>|  Sentence \\/ Sentence</td></tr>\n<tr><td></td><td>|  Sentence -> Sentence</td></tr>\n<tr><td></td><td>|  Sentence <-> Sentence</td></tr>\n<tr><td>Operator Precedence</td><td>~, /\\, \\/, ->, <-></td></tr>\n</table>\n</i>\nComplex sentences may be evaluated with <b>truth tables</b>. These expand all the possibilities for truth values from a component, e.g. given a statement not \n<i>P<sub>1</sub></i> /\\ (<i>P<sub>2</sub></i> \\/ <i>P<sub>3</sub></i>), it is known that if the statements are <i>true</i>, <i>false</i>, and <i>true</i> respectively\nthen the statement evaluates to <i>true</i> /\\ (<i>false</i> \\/ <i>true</i>) = <i>true</i> /\\ <i>true</i> = <i>true</i>.\n<br /><br />\nAn inference algorithm should be implemented to be both <b>sound</b> and <b>complete</b>, meaning that everything expanded is entailed by the KB (thereby, preserving\nthe truthfulness of the KB) and that it expands all the possibilities for a given KB.\n\n## AIMA 7.5 Propositional Theorem Proving\n<b>Theorem proving</b> or applying rules of inference directly to sentences is oftentimes more efficient than model checking. Three related key concepts:\n<ul><li><b>logical equivalence</b>: sentences are logically equivalent if they are true in the same set of models.</li>\n<li><b>validity</b>: true in <i>all</i> models, also the same as a <i>tautology</i></li>\n<li><b>satisfiability</b>: if a statement is satisfied by or true in <i>some</i> model</li>\n</ul>\nIt can be said that a sentence is viable if and only if its negation is unsatisfiable and vice versa. This is also a form of proof by <b>refutation</b> or <b>contradiction</b>.\n<br /><br />\nSome <b>inference rules</b> that can be applied to derivve a proof include <b>Modus ponus</b> or inferring the implication if its condition is met, the <b>And-elimination</b>\nwhich states any of the conjuncts can be inferred, etc. <b>Resolution</b> is a rule that concludes a statement based on the negation of its other dijuncts.\n<br /><br />\nTo get a sentence into <b>conjunctive normal form</b> (<b>CNF</b>) we do the following:\n<ol><li>Eliminate <->, replacing <i>A <-> B</i> with (<i>A -> B</i>) /\\ (<i>B -> A</i>)</li>\n<li>Eliminate ->, replacing <i>A -> B</i> with <i>~A</i> \\/ <i>B</i></li>\n<li>~ can only apply in literals, \"move ~ inwards\" by using De Morgan and double-negation elimination</li>\n<li>Distribute \\/ over /\\ wherever possible</li></ol>\n<b>Resolution closure</b> <i>RC(s)</i> is the set of all caluses derivable by a repeated application of resolution rule to the caluses in a set <i>S</i>. The completeness\nfor resolution in propositional logic is via the <b>ground resolution theorem</b>: If a set of clauses is unsatisfiable, then the resolution closure of those clauses\ncontains the empty clause.\n<br /><br />\nSometimes, the full power of resolution is not needed. Some KBs have restrictions which enable them to use a more efficient inference algorithm. For example there are\n<b>definite clause</b>s which are a dijunction of literals in which <i>exactly one</i> is positive. Even more general is the <b>horn clause</b> which is a dijunction in\nwhich <i>at most</i> one is <i>positive</i> - these include <b>goal caluses</b> in which none are true. Another is the <i>k</i>-CNF sentence which is a CNF sentence with\nat most <i>k</i> literals. These are interesting because: \n<br /><br />\nThe process of <b>forward chaining</b> is <b>data driven</b> because it begins from known facts and then adds statements until nof urther inferences can be made. It is both\n<b>sound since each inference is an application of Modus Ponus but also <b>complete</b> since if there were other sentences that could be added but were not, the\nalgorithm would add those before terminating. Backwards chaining similarly works backwards from the query to find the implications whose conclusion is <i>q</i>. This is a\nform of <b>goal-directed reasoning</b> and is often linear with size of the KB since it only touches relevant facts.\n\n## Module 7 Lecture Notes\n Your notes here \n\n## Module 7 Definitions\n* **Knowledge base** - a set of sentences that comprise the central component of a knowledge-based agent\n* **Sentence** - a component statement of a knowledge base, expressed in knowledge rperesentation language\n* **Knowledge representation language** - a logical expression that represents some assertion about the real world in a knowledge base\n* **Axiom** - a sentence that is given as truthful, rather than being derived from other sentences\n* **Inference** - the process of deriving a new sentence based on old sentences\n* **Background knowledge** - knowledge that is known beforehand\n* **Knowledge level** - the most abstract level in which the agent and its goals are defined\n* **Implementation level** - the specifics to how an agent is implemented\n* **Declarative approach** - giving an agent known statements until the agent knows how to behave\n* **Procedural approach** - explicitly defining how ana gent should behave\n* **Performance measure** - a function that defines how an agent has performed at a task\n* **Environment** - the available space for the agent, e.g. size of the model, placement of objects, etc.\n* **Actuators** - behaviors that an agent can perform\n* **Sensors** - precepts that are given to an agent to infer more about its environment\n* **Syntax** - the rules that define how sentences can be formed in a representation language\n* **Semantics** - meaning of a sentence, defines its truth value in a possible world\n* **Model** - a mathematical abstraction with a fixed truth value\n* **Entailment** - a sentence entails another if the other statement logically follows from it\n* **Logical inference** - applying entailments to derive new conclusions\n* **Model checking** - a specific logical inference algorithm that enumerates all possible models to validate the truthness of a sentence\n* **Grounding** - the connection between sentences and the real world\n* **Soundness / truth preserving** - entailed sentences that are derived do not invalidate the truthness of other senteces in knowledge base\n* **Completness** - inference algorithm derives all entailed sentences\n* **Learning** - the process constructing general rules from existing sentences\n* **Propositional logic** - logic in which atominc sentences are denoted by a single propositional symbol\n* **Atomic sentence** - a statement or claim about the world\n* **Complex sentence** - a sentence that combines multiple atomic sentences\n* **Logical connectives** - operators that connect different atomic sentences\n* **Literal** - an atomic sentence (a positive literal) or a negated atomic sentence (a negative literal\n* **Negation** - the inverse value of\n* **Conjunction** - logical and\n* **Conjuncts** - the components in a conjunction\n* **Disjunction** - logical or\n* **Disjuncts** - the components in a disjunction\n* **Implication** - an atomic sentence that follows from another atomic sentence\n* **Premise** - or antecedent, the first part of an implication or biconditional\n* **Conclusion** - or consequent, the implied part of an implication\n* **Biconditional** - logical if and only if\n* **Truth value** - if an atomic sentence is <i>true</i> or <i>false</i>\n* **Truth tables** - an expression of propositional logic rules and their truth values based on the atomic statements\n* **Sound** - an algorithm that implements the definition of entailment directly\n* **Complete** - an algorithm that derives every possible statement in the KB\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "metadata": {}
        }
      ]
    }
  ]
}