{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# CVXPY Tutorial\n",
        "\n",
        "*Pierfrancesco Beneventano*\n",
        "\n",
        "\\\n",
        "\n",
        "ORF363/COS323 Fall 2023\n",
        "\n",
        "Prof. Amir Ali Ahmadi\n",
        "\n",
        "TA: Beneventano, Chaudhry, Hua, Li, Lok\n",
        "\n",
        "---\n",
        "\n",
        "\\\n",
        "\n",
        "Welcome to the CVXPY tutorial! In this session, we'll explore how to formulate and solve convex optimization problems using CVXPY in Python. If you're familiar with MATLAB's CVX, you'll find many similarities, but also some differences in syntax and functionality.\n",
        "\n",
        "CVXPY is a Python library designed to make it easy to construct and solve convex optimization problems. It provides a simple interface to define objectives, constraints, and solve optimization problems, all while writing code that closely mirrors the mathematical formulation.\n",
        "\n",
        "Important info: One of the people that developed and maintain it is Prof. Bartolomeo Stellato at ORFE!\n",
        "\n"
      ],
      "metadata": {
        "id": "E0xjDPZzb8pf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Installing of CVXPY\n",
        "\n",
        "Before we dive into the examples, let's ensure CVXPY is installed.\n"
      ],
      "metadata": {
        "id": "y9axXKIlclwY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install cvxpy\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7zYtgfxkdEzz",
        "outputId": "b704fa14-9a8e-4e0e-8a37-55eebda7797b"
      },
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: cvxpy in /usr/local/lib/python3.10/dist-packages (1.3.2)\n",
            "Requirement already satisfied: osqp>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from cvxpy) (0.6.2.post8)\n",
            "Requirement already satisfied: ecos>=2 in /usr/local/lib/python3.10/dist-packages (from cvxpy) (2.0.12)\n",
            "Requirement already satisfied: scs>=1.1.6 in /usr/local/lib/python3.10/dist-packages (from cvxpy) (3.2.3)\n",
            "Requirement already satisfied: numpy>=1.15 in /usr/local/lib/python3.10/dist-packages (from cvxpy) (1.23.5)\n",
            "Requirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from cvxpy) (1.11.2)\n",
            "Requirement already satisfied: setuptools>65.5.1 in /usr/local/lib/python3.10/dist-packages (from cvxpy) (67.7.2)\n",
            "Requirement already satisfied: qdldl in /usr/local/lib/python3.10/dist-packages (from osqp>=0.4.1->cvxpy) (0.1.7.post0)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "You will generally just need to run pip install for cvxpy. That said you may want to use different solvers in the future, to install them or if you have any issue with the installation a useful link is\n",
        "[this one](https://www.cvxpy.org/install/index.html).\n",
        "\n",
        "\n",
        "When you run this command:\n",
        "\n",
        "- *pip* will search for the CVXPY package on the Python Package Index (PyPI).\n",
        "It will download the CVXPY package.\n",
        "It will install CVXPY and its dependencies.\n",
        "\n",
        "\n",
        "Things to Note:\n",
        "\n",
        "- **Dependencies:** CVXPY has several dependencies that will also be installed. This includes the default solvers that CVXPY uses, as well as other required libraries.\n",
        "\n",
        "- **Additional Solvers:** While CVXPY comes with some default solvers, there might be specific solvers that you want to install separately.\n",
        "\n",
        "- **Virtual Environments:** It's a good practice to use virtual environments in Python to manage dependencies. This ensures that different projects don't interfere with each other in terms of the versions of packages they use. Ask chatGPT or us about it!\n",
        "\n",
        "- **Jupyter Notebooks or Google Colab:** If you're using Jupyter Notebooks for the tutorial, students can install CVXPY directly from the notebook using !pip install cvxpy. If you're using Google Colab, CVXPY is already pre-installed.\n",
        "\n"
      ],
      "metadata": {
        "id": "wzMn5aitdRN4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Basic Problem Formulation\n",
        "\n",
        "Let's start with a simple linear programming problem to understand the basics of CVXPY.\n",
        "\n",
        "PS: These are the same problems as the MATLAB tutorial!"
      ],
      "metadata": {
        "id": "ssI7Eey_f20r"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import cvxpy as cp\n",
        "\n",
        "# Define variables\n",
        "x1 = cp.Variable()\n",
        "x2 = cp.Variable()\n",
        "\n",
        "# Define objective\n",
        "objective = cp.Maximize(140*x1 + 235*x2)\n",
        "\n",
        "# Define constraints\n",
        "constraints = [\n",
        "    x1 >= 0,\n",
        "    x2 >= 0,\n",
        "    x1 + x2 <= 180,\n",
        "    x1 + 2*x2 <= 240,\n",
        "    0.3*x1 + 0.1*x2 <= 30\n",
        "]\n",
        "\n",
        "# Formulate the problem\n",
        "problem = cp.Problem(objective, constraints)\n",
        "\n",
        "# Solve the problem\n",
        "problem.solve()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0qRCkkHmgO2c",
        "outputId": "33b012d1-b48e-4f32-f209-de86345f8824"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "29819.999991399833"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Display results\n",
        "x1_value = x1.value\n",
        "x2_value = x2.value\n",
        "optimal_value = problem.value\n",
        "\n",
        "x1_value, x2_value, optimal_value\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O44XNM7QgnJi",
        "outputId": "8ce8ec06-6327-42d1-9ba2-6d9d09e5c7ba"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array(71.99999989), array(84.00000003), 29819.999991399833)"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Transition from MATLAB's CVX\n",
        "\n",
        "For those familiar with MATLAB's CVX, the structure of problem definition in CVXPY should feel familiar. However, there are some syntactical differences due to the inherent differences between MATLAB and Python.\n",
        "\n",
        "Here's a brief comparison:\n",
        "\n",
        "- MATLAB uses `cvx_begin` and `cvx_end` to delineate the problem. In CVXPY, we define the problem using Python constructs and encapsulate it in a `Problem` object.\n",
        "- Matrix multiplication in MATLAB uses `*`, while in Python (and CVXPY), we use the `@` operator.\n",
        "- After solving in CVX, variables hold their optimal values. In CVXPY, we use the `.value` attribute of a variable to access its optimal value.\n",
        "\n",
        "Let's explore this better.\n"
      ],
      "metadata": {
        "id": "Lqqq_k0Llbaj"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Begin and End\n",
        "\n",
        "In MATLAB's CVX, the `cvx_begin` and `cvx_end` commands are used to delineate the convex optimization problem specification. Everything between these two commands is treated as part of the CVX problem definition.\n",
        "\n",
        "#### Usage of `cvx_begin` and `cvx_end`:\n",
        "\n",
        "- **cvx_begin**: Marks the beginning of the CVX problem specification. It signals to MATLAB that the following lines will define a convex optimization problem using CVX's domain-specific language.\n",
        "\n",
        "- **cvx_end**: Marks the end of the CVX problem specification. When MATLAB encounters the `cvx_end` command, it processes the problem, sends it to the solver, and retrieves the solution.\n",
        "\n",
        "After the `cvx_end` command is executed, the CVX variables become standard MATLAB variables. Their values are set to the optimal values found by the solver. So, if you had a variable `x` defined within the `cvx_begin` and `cvx_end` block, after `cvx_end`, `x` will be a regular MATLAB variable containing the optimal value.\n",
        "\n",
        "### CVXPY Syntax:\n",
        "\n",
        "CVXPY does not use a similar `begin` and `end` syntax. Instead, in CVXPY:\n",
        "\n",
        "1. Variables, objectives, and constraints are defined using CVXPY functions and standard Python syntax.\n",
        "2. The optimization problem is encapsulated in a `Problem` object.\n",
        "3. The `solve` method of the `Problem` object is called to solve the problem.\n",
        "\n",
        "The reason for this difference is largely due to the design philosophies of the two environments. MATLAB's scripting nature allows for domain-specific language constructs like `cvx_begin` and `cvx_end` to be interwoven directly into the script. Python, being more object-oriented, encourages encapsulation, and CVXPY's design reflects this by encapsulating optimization problems within objects.\n",
        "\n",
        "In CVXPY, once the problem is solved, the `value` attribute of a CVXPY variable can be accessed to get the optimal value. However, the CVXPY variable remains an object of type `Variable` and doesn't become a standard Python variable. If you want to extract its value as a standard Python variable, you'd do something like:\n",
        "\n",
        "```python\n",
        "x_optimal_value = x.value\n",
        "```\n",
        "\n",
        "In summary, while CVX uses a domain-specific language embedded within MATLAB scripts to define and solve problems, CVXPY uses a more object-oriented approach consistent with Python's design principles."
      ],
      "metadata": {
        "id": "0PFqvtjyk-hl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Verbose\n",
        "\n",
        "CVXPY provides a more concise output by default compared to MATLAB's CVX. However, you can make CVXPY's output more verbose by adjusting the solver's verbosity settings.\n",
        "\n",
        "To enable more verbose output, you can set the verbose argument to True when calling the solve method of a Problem object.\n",
        "\n",
        "Here's how you can modify the previous code snippets to include more verbose output:"
      ],
      "metadata": {
        "id": "j37bfQLYhTeF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Same as above but verbose\n",
        "\n",
        "import cvxpy as cp\n",
        "\n",
        "# Define variables\n",
        "x1 = cp.Variable()\n",
        "x2 = cp.Variable()\n",
        "\n",
        "# Define objective\n",
        "objective = cp.Maximize(140*x1 + 235*x2)\n",
        "\n",
        "# Define constraints\n",
        "constraints = [\n",
        "    x1 >= 0,\n",
        "    x2 >= 0,\n",
        "    x1 + x2 <= 180,\n",
        "    x1 + 2*x2 <= 240,\n",
        "    0.3*x1 + 0.1*x2 <= 30\n",
        "]\n",
        "\n",
        "# Formulate the problem\n",
        "problem = cp.Problem(objective, constraints)\n",
        "\n",
        "# Solve the problem\n",
        "problem.solve(verbose = True)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "52865c2d-9538-42b9-a94d-18359b52e2a3",
        "id": "tEGNFlDIhavw"
      },
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "===============================================================================\n",
            "                                     CVXPY                                     \n",
            "                                     v1.3.2                                    \n",
            "===============================================================================\n",
            "(CVXPY) Sep 28 04:38:35 PM: Your problem has 2 variables, 5 constraints, and 0 parameters.\n",
            "(CVXPY) Sep 28 04:38:35 PM: It is compliant with the following grammars: DCP, DQCP\n",
            "(CVXPY) Sep 28 04:38:35 PM: (If you need to solve this problem multiple times, but with different data, consider using parameters.)\n",
            "(CVXPY) Sep 28 04:38:35 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                  Compilation                                  \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:35 PM: Compiling problem (target solver=ECOS).\n",
            "(CVXPY) Sep 28 04:38:35 PM: Reduction chain: FlipObjective -> Dcp2Cone -> CvxAttr2Constr -> ConeMatrixStuffing -> ECOS\n",
            "(CVXPY) Sep 28 04:38:35 PM: Applying reduction FlipObjective\n",
            "(CVXPY) Sep 28 04:38:35 PM: Applying reduction Dcp2Cone\n",
            "(CVXPY) Sep 28 04:38:35 PM: Applying reduction CvxAttr2Constr\n",
            "(CVXPY) Sep 28 04:38:35 PM: Applying reduction ConeMatrixStuffing\n",
            "(CVXPY) Sep 28 04:38:35 PM: Applying reduction ECOS\n",
            "(CVXPY) Sep 28 04:38:35 PM: Finished problem compilation (took 3.167e-02 seconds).\n",
            "-------------------------------------------------------------------------------\n",
            "                                Numerical solver                               \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:35 PM: Invoking solver ECOS  to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                    Summary                                    \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:35 PM: Problem status: optimal\n",
            "(CVXPY) Sep 28 04:38:35 PM: Optimal value: 2.982e+04\n",
            "(CVXPY) Sep 28 04:38:35 PM: Compilation took 3.167e-02 seconds\n",
            "(CVXPY) Sep 28 04:38:35 PM: Solver (including time spent in interface) took 2.029e-04 seconds\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "29819.999991399833"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### The Attributes\n",
        "\n",
        "CVXPY variables have several attributes and methods associated with them. Here are some of the key attributes of a CVXPY variable:\n",
        "\n",
        "1. **`.value`**: As already used above, this attribute holds the optimal value of the variable after the problem has been solved.\n",
        "\n",
        "2. **`.name()`**: If you've named your variable during creation (e.g., `x = cp.Variable(name=\"x\")`), you can retrieve this name using the `.name` attribute.\n",
        "\n",
        "3. **`.shape`**: This attribute returns the shape of the variable. For instance, if the variable is a matrix of size \\(m \\times n\\), `.shape` will return `(m, n)`.\n",
        "\n",
        "4. **`.size`**: Returns the total number of elements in the variable.\n",
        "\n",
        "5. **`.is_positive()`**: Returns a constraint that the variable is elementwise positive.\n",
        "\n",
        "6. **`.is_negative()`**: Returns a constraint that the variable is elementwise negative.\n",
        "\n",
        "7. **`.is_nonpositive()`**: Returns a constraint that the variable is elementwise non-positive.\n",
        "\n",
        "8. **`.is_nonnegative()`**: Returns a constraint that the variable is elementwise non-negative.\n",
        "\n",
        "9. **`.T`**: Returns the transpose of the variable if it's a matrix.\n",
        "\n",
        "10. **`.value`**: The current value of the variable. This is set after solving a problem.\n",
        "\n",
        "11. **`.parameters()`**: A list of parameters associated with the variable.\n",
        "\n",
        "12. **`.log_log_curvature`** and **`.log_log_convex`**: These attributes are related to the curvature of the variable in the log-log space, which can be useful in advanced applications.\n",
        "\n",
        "13. **`.P`**, **`.q`**, **`.r`**: These attributes are related to the quadratic form representation of the variable.\n",
        "\n",
        "These are some of the main attributes and methods, but there are others, especially when you get into more advanced uses of CVXPY. If you want a comprehensive list and detailed explanations, the [official CVXPY documentation](https://www.cvxpy.org/) is a great resource."
      ],
      "metadata": {
        "id": "gx8Fjpw7hiv2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Access and print attributes/methods\n",
        "print(\"\\nAttributes after verbose solve:\")\n",
        "print(f\"x1 name: {x1.name()}\")\n",
        "print(f\"x1 shape: {x1.shape}\")\n",
        "print(f\"x1 value: {x1.value}\")\n",
        "print(f\"x1 value: {x1.parameters()}\\n\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TZhVZb7Sjl3k",
        "outputId": "0da152f0-78a2-4b90-83e6-ffad8777edb4"
      },
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Attributes after verbose solve:\n",
            "x1 name: var1196\n",
            "x1 shape: ()\n",
            "x1 value: 71.999999890297\n",
            "x1 value: []\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Least Square"
      ],
      "metadata": {
        "id": "NYckfV9wl6kj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import cvxpy as cp\n",
        "import numpy as np\n",
        "\n",
        "# Data generation\n",
        "m, n = 16, 8\n",
        "A = np.random.randn(m, n)\n",
        "b = np.random.randn(m).reshape(-1)  # Reshape b to be a 1D array\n",
        "\n",
        "# Variable definition\n",
        "x = cp.Variable(n)\n",
        "\n",
        "# 1. Minimize the convex quadratic\n",
        "objective1 = cp.Minimize(cp.quad_form(x, A.T @ A) - 2*b.T @ A @ x + cp.sum_squares(b))\n",
        "problem1 = cp.Problem(objective1)\n",
        "problem1.solve(verbose = True)\n",
        "optimal_value1 = problem1.value\n",
        "\n",
        "# 2. Minimize the norm of an affine function\n",
        "objective2 = cp.Minimize(cp.norm(A @ x - b))\n",
        "problem2 = cp.Problem(objective2)\n",
        "problem2.solve(verbose = True)\n",
        "optimal_value2 = problem2.value\n",
        "\n",
        "# # 3. Maximize the norm (This will produce an error as expected)\n",
        "# try:\n",
        "#     objective3 = cp.Maximize(cp.norm(A @ x - b))\n",
        "#     problem3 = cp.Problem(objective3)\n",
        "#     problem3.solve()\n",
        "#     optimal_value3 = problem3.value\n",
        "# except Exception as e:\n",
        "#     optimal_value3 = str(e)\n",
        "\n",
        "# 4. LASSO-type problem\n",
        "objective4 = cp.Minimize(cp.norm(A @ x - b) + cp.norm(x, 1))\n",
        "problem4 = cp.Problem(objective4)\n",
        "problem4.solve(verbose = True)\n",
        "optimal_value4 = problem4.value\n",
        "\n",
        "optimal_value1, optimal_value2, optimal_value4\n"
      ],
      "metadata": {
        "id": "sFXJzq0zkMOi",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5d88b4d1-fc71-40d5-9c44-425081fc14ea"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "===============================================================================\n",
            "                                     CVXPY                                     \n",
            "                                     v1.3.2                                    \n",
            "===============================================================================\n",
            "(CVXPY) Sep 28 04:38:36 PM: Your problem has 8 variables, 0 constraints, and 0 parameters.\n",
            "(CVXPY) Sep 28 04:38:36 PM: It is compliant with the following grammars: DCP, DQCP\n",
            "(CVXPY) Sep 28 04:38:36 PM: (If you need to solve this problem multiple times, but with different data, consider using parameters.)\n",
            "(CVXPY) Sep 28 04:38:36 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                  Compilation                                  \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:36 PM: Compiling problem (target solver=OSQP).\n",
            "(CVXPY) Sep 28 04:38:36 PM: Reduction chain: CvxAttr2Constr -> Qp2SymbolicQp -> QpMatrixStuffing -> OSQP\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction CvxAttr2Constr\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction Qp2SymbolicQp\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction QpMatrixStuffing\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction OSQP\n",
            "(CVXPY) Sep 28 04:38:36 PM: Finished problem compilation (took 2.515e-02 seconds).\n",
            "-------------------------------------------------------------------------------\n",
            "                                Numerical solver                               \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:36 PM: Invoking solver OSQP  to obtain a solution.\n",
            "-----------------------------------------------------------------\n",
            "           OSQP v0.6.2  -  Operator Splitting QP Solver\n",
            "              (c) Bartolomeo Stellato,  Goran Banjac\n",
            "        University of Oxford  -  Stanford University 2021\n",
            "-----------------------------------------------------------------\n",
            "problem:  variables n = 8, constraints m = 0\n",
            "          nnz(P) + nnz(A) = 36\n",
            "settings: linear system solver = qdldl,\n",
            "          eps_abs = 1.0e-05, eps_rel = 1.0e-05,\n",
            "          eps_prim_inf = 1.0e-04, eps_dual_inf = 1.0e-04,\n",
            "          rho = 1.00e-01 (adaptive),\n",
            "          sigma = 1.00e-06, alpha = 1.60, max_iter = 10000\n",
            "          check_termination: on (interval 25),\n",
            "          scaling: on, scaled_termination: off\n",
            "          warm start: on, polish: on, time_limit: off\n",
            "\n",
            "iter   objective    pri res    dua res    rho        time\n",
            "   1  -5.1772e+00   0.00e+00   9.04e+00   1.00e-01   8.51e-05s\n",
            "  25  -8.0893e+00   0.00e+00   4.28e-05   1.00e-01   1.47e-04s\n",
            "plsh  -8.0893e+00   0.00e+00   3.35e-15   --------   2.21e-04s\n",
            "\n",
            "status:               solved\n",
            "solution polish:      successful\n",
            "number of iterations: 25\n",
            "optimal objective:    -8.0893\n",
            "run time:             2.21e-04s\n",
            "optimal rho estimate: 1.00e-06\n",
            "\n",
            "-------------------------------------------------------------------------------\n",
            "                                    Summary                                    \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:36 PM: Problem status: optimal\n",
            "(CVXPY) Sep 28 04:38:36 PM: Optimal value: 8.664e+00\n",
            "(CVXPY) Sep 28 04:38:36 PM: Compilation took 2.515e-02 seconds\n",
            "(CVXPY) Sep 28 04:38:36 PM: Solver (including time spent in interface) took 2.630e-03 seconds\n",
            "===============================================================================\n",
            "                                     CVXPY                                     \n",
            "                                     v1.3.2                                    \n",
            "===============================================================================\n",
            "(CVXPY) Sep 28 04:38:36 PM: Your problem has 8 variables, 0 constraints, and 0 parameters.\n",
            "(CVXPY) Sep 28 04:38:36 PM: It is compliant with the following grammars: DCP, DQCP\n",
            "(CVXPY) Sep 28 04:38:36 PM: (If you need to solve this problem multiple times, but with different data, consider using parameters.)\n",
            "(CVXPY) Sep 28 04:38:36 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                  Compilation                                  \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:36 PM: Compiling problem (target solver=ECOS).\n",
            "(CVXPY) Sep 28 04:38:36 PM: Reduction chain: Dcp2Cone -> CvxAttr2Constr -> ConeMatrixStuffing -> ECOS\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction Dcp2Cone\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction CvxAttr2Constr\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction ConeMatrixStuffing\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction ECOS\n",
            "(CVXPY) Sep 28 04:38:36 PM: Finished problem compilation (took 3.133e-02 seconds).\n",
            "-------------------------------------------------------------------------------\n",
            "                                Numerical solver                               \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:36 PM: Invoking solver ECOS  to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                    Summary                                    \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:36 PM: Problem status: optimal\n",
            "(CVXPY) Sep 28 04:38:36 PM: Optimal value: 2.944e+00\n",
            "(CVXPY) Sep 28 04:38:36 PM: Compilation took 3.133e-02 seconds\n",
            "(CVXPY) Sep 28 04:38:36 PM: Solver (including time spent in interface) took 5.019e-04 seconds\n",
            "===============================================================================\n",
            "                                     CVXPY                                     \n",
            "                                     v1.3.2                                    \n",
            "===============================================================================\n",
            "(CVXPY) Sep 28 04:38:36 PM: Your problem has 8 variables, 0 constraints, and 0 parameters.\n",
            "(CVXPY) Sep 28 04:38:36 PM: It is compliant with the following grammars: DCP, DQCP\n",
            "(CVXPY) Sep 28 04:38:36 PM: (If you need to solve this problem multiple times, but with different data, consider using parameters.)\n",
            "(CVXPY) Sep 28 04:38:36 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                  Compilation                                  \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:36 PM: Compiling problem (target solver=ECOS).\n",
            "(CVXPY) Sep 28 04:38:36 PM: Reduction chain: Dcp2Cone -> CvxAttr2Constr -> ConeMatrixStuffing -> ECOS\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction Dcp2Cone\n",
            "(CVXPY) Sep 28 04:38:36 PM: Applying reduction CvxAttr2Constr\n",
            "(CVXPY) Sep 28 04:38:37 PM: Applying reduction ConeMatrixStuffing\n",
            "(CVXPY) Sep 28 04:38:37 PM: Applying reduction ECOS\n",
            "(CVXPY) Sep 28 04:38:37 PM: Finished problem compilation (took 3.549e-02 seconds).\n",
            "-------------------------------------------------------------------------------\n",
            "                                Numerical solver                               \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:37 PM: Invoking solver ECOS  to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                    Summary                                    \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:37 PM: Problem status: optimal\n",
            "(CVXPY) Sep 28 04:38:37 PM: Optimal value: 3.879e+00\n",
            "(CVXPY) Sep 28 04:38:37 PM: Compilation took 3.549e-02 seconds\n",
            "(CVXPY) Sep 28 04:38:37 PM: Solver (including time spent in interface) took 5.064e-04 seconds\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(8.664296684841993, 2.943517739855153, 3.8792503199899873)"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Constrained Norm Minimization\n",
        "\n",
        "In this exercise, we'll minimize the norm of `A*x - b` subject to some constraints.\n"
      ],
      "metadata": {
        "id": "jIdksDuzocs8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import cvxpy as cp\n",
        "import numpy as np\n",
        "\n",
        "# Data generation\n",
        "m, n = 16, 8\n",
        "A = np.random.randn(m, n)\n",
        "b = np.random.randn(m).reshape(-1)\n",
        "\n",
        "p = 4\n",
        "C = np.random.randn(p, n)\n",
        "d = np.random.randn(p).reshape(-1)\n",
        "\n",
        "# Variable definition\n",
        "x = cp.Variable(n)\n",
        "\n",
        "# Objective\n",
        "objective = cp.Minimize(cp.norm(A @ x - b))\n",
        "\n",
        "# Constraints\n",
        "constraints = [\n",
        "    C @ x == d,       # Equality constraint\n",
        "    x >= 0.1,        # Elementwise constraint\n",
        "    cp.norm(x, \"inf\") <= 100  # Infinity norm constraint\n",
        "]\n",
        "\n",
        "# Problem formulation\n",
        "problem = cp.Problem(objective, constraints)\n",
        "\n",
        "# Solve\n",
        "problem.solve()\n",
        "\n",
        "# Display results\n",
        "x_value = x.value\n",
        "optimal_value = problem.value\n",
        "\n",
        "x_value, optimal_value\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-Mmni3y9mBTf",
        "outputId": "ee792aed-8f70-44c0-fbb9-7f33c6ff2017"
      },
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([0.49435871, 1.95177733, 0.13947031, 0.1       , 0.1       ,\n",
              "        0.1       , 2.4101929 , 0.1       ]),\n",
              " 11.754037412675553)"
            ]
          },
          "metadata": {},
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This code will:\n",
        "\n",
        "1. Minimize the 2-norm of $A x - b$.\n",
        "2. Subject to the constraints that $C x = d$, $x \\geq 0.1$ elementwise, and the infinity norm of $x$ is less than or equal to 100.\n",
        "\n",
        "The results for the problem are stored in `x_value` and `optimal_value`."
      ],
      "metadata": {
        "id": "ChFKxd6Aprl-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Infeasible Problem\n",
        "\n",
        "Let's now explore a problem that's infeasible. An infeasible problem is one where no solution exists that satisfies all the constraints.\n"
      ],
      "metadata": {
        "id": "rEFULB-nogRV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import cvxpy as cp\n",
        "\n",
        "# Variable definition\n",
        "x1 = cp.Variable()\n",
        "x2 = cp.Variable()\n",
        "\n",
        "# Objective\n",
        "objective = cp.Minimize(x1 + x2)\n",
        "\n",
        "# Constraints\n",
        "constraints = [\n",
        "    x1 + x2 <= -5,\n",
        "    x1**2 + x2**2 <= 1  # This constraint makes the problem infeasible\n",
        "]\n",
        "\n",
        "# Problem formulation\n",
        "problem = cp.Problem(objective, constraints)\n",
        "\n",
        "# Solve\n",
        "problem_status = problem.solve(verbose = True)\n",
        "\n",
        "# Display results\n",
        "problem_status, x1.value, x2.value\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tr0rhonrogon",
        "outputId": "f947c83a-90d2-47b9-e900-b9edf80d56ea"
      },
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "===============================================================================\n",
            "                                     CVXPY                                     \n",
            "                                     v1.3.2                                    \n",
            "===============================================================================\n",
            "(CVXPY) Sep 28 04:38:38 PM: Your problem has 2 variables, 2 constraints, and 0 parameters.\n",
            "(CVXPY) Sep 28 04:38:38 PM: It is compliant with the following grammars: DCP, DQCP\n",
            "(CVXPY) Sep 28 04:38:38 PM: (If you need to solve this problem multiple times, but with different data, consider using parameters.)\n",
            "(CVXPY) Sep 28 04:38:38 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                  Compilation                                  \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:38 PM: Compiling problem (target solver=ECOS).\n",
            "(CVXPY) Sep 28 04:38:38 PM: Reduction chain: Dcp2Cone -> CvxAttr2Constr -> ConeMatrixStuffing -> ECOS\n",
            "(CVXPY) Sep 28 04:38:38 PM: Applying reduction Dcp2Cone\n",
            "(CVXPY) Sep 28 04:38:38 PM: Applying reduction CvxAttr2Constr\n",
            "(CVXPY) Sep 28 04:38:38 PM: Applying reduction ConeMatrixStuffing\n",
            "(CVXPY) Sep 28 04:38:38 PM: Applying reduction ECOS\n",
            "(CVXPY) Sep 28 04:38:38 PM: Finished problem compilation (took 5.932e-02 seconds).\n",
            "-------------------------------------------------------------------------------\n",
            "                                Numerical solver                               \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:38 PM: Invoking solver ECOS  to obtain a solution.\n",
            "-------------------------------------------------------------------------------\n",
            "                                    Summary                                    \n",
            "-------------------------------------------------------------------------------\n",
            "(CVXPY) Sep 28 04:38:38 PM: Problem status: infeasible\n",
            "(CVXPY) Sep 28 04:38:38 PM: Optimal value: inf\n",
            "(CVXPY) Sep 28 04:38:38 PM: Compilation took 5.932e-02 seconds\n",
            "(CVXPY) Sep 28 04:38:38 PM: Solver (including time spent in interface) took 2.323e-03 seconds\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(inf, None, None)"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This code will:\n",
        "\n",
        "1. Minimize the sum of $x_1$ and $x_2$.\n",
        "2. Subject to the constraints $x_1 + x_2 \\leq -5$ and $x_1^2 + x_2^2 \\leq 1$.\n",
        "\n",
        "The problem is infeasible due to the constraints. The code will return the problem status (which should be \"infeasible\") and the values of `x1` and `x2` (which will be `None` since the problem is infeasible)."
      ],
      "metadata": {
        "id": "ufUOALbKpU5-"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "c_VHeAoIooPT"
      },
      "execution_count": 48,
      "outputs": []
    }
  ]
}