probability-uncertainty.html (6861B)
1 <!DOCTYPE html> 2 <html> 3 <head> 4 <script type="text/javascript" async src="https://cdn.jsdelivr.net/gh/mathjax/MathJax@2.7.5/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script> 5 <link rel="Stylesheet" type="text/css" href="style.css"> 6 <title>probability-uncertainty</title> 7 <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 8 </head> 9 <body> 10 11 <div id="Probability and Uncertainty"><h1 id="Probability and Uncertainty">Probability and Uncertainty</h1></div> 12 <p> 13 one often has to deal with info that is underspecified, incomplete, vague, etc. 14 </p> 15 16 <p> 17 logic by itself is not sufficient for these problems. 18 </p> 19 20 <div id="Probability and Uncertainty-Vagueness: Fuzzy Set Theory"><h2 id="Vagueness: Fuzzy Set Theory">Vagueness: Fuzzy Set Theory</h2></div> 21 <p> 22 model theory often based on set theory 23 </p> 24 25 <p> 26 fuzzy set theory allows something to be <em>to some degree</em> an element of a set 27 </p> 28 29 <p> 30 dominant approach to vagueness (mostly because wtf else can you do) 31 </p> 32 33 <div id="Probability and Uncertainty-Vagueness: Fuzzy Set Theory-Fuzzy sets"><h3 id="Fuzzy sets">Fuzzy sets</h3></div> 34 <ul> 35 <li> 36 universe U, object x ∈ U 37 38 <li> 39 membership function for fuzzy set A is defined to be function \(f_A\) from U to [0,1]: 40 41 <ul> 42 <li> 43 \(f_A(x)=y\): x is a member of A to degree y 44 45 <li> 46 \(f_A(x)=1\): x is certainly member of A 47 48 <li> 49 \(f_A(x)=0\): x is certainly not a member of A 50 51 <li> 52 \({x | f_A(x)>0}\): support of A 53 54 </ul> 55 </ul> 56 57 <p> 58 modifiers (hedges) 59 </p> 60 61 <p> 62 <img src="img/modifiers-hedges.png" alt="Example of modifiers' effects on graphs" /> 63 </p> 64 65 <p> 66 operations on fuzzy sets: 67 </p> 68 <ul> 69 <li> 70 complement: \(f_{\sim{A}}(x)=1-f_A(x)\) 71 72 <li> 73 union: \(f_{A\cup B}(x)=\max(f_A(x),f_B(x))\) 74 75 <li> 76 intersection: \(f_{A\cap B}(x)=\min(f_A(x), f_B(x))\) 77 78 <li> 79 subset: \(A\subset B \iff \forall{x}(f_A(x)\leq f_B(x))\) 80 81 </ul> 82 83 <p> 84 semantics -- multivalued fuzzy logic 85 </p> 86 <ul> 87 <li> 88 v(¬ A) = 1-v(A) 89 90 <li> 91 v(A ∨ B) = max(v(A), v(B)) 92 93 <li> 94 v(A ∧ B) = min(v(A), v(B)) 95 96 <li> 97 v(A → B) = min(1, 1 - v(A) + v(B)) 98 99 </ul> 100 101 <div id="Probability and Uncertainty-Vagueness: Fuzzy Set Theory-Fuzzy relations"><h3 id="Fuzzy relations">Fuzzy relations</h3></div> 102 <p> 103 fuzzy sets can denote fuzzy relations between objects. e.g. approximately equal, close to, much larger than, etc. 104 </p> 105 106 <p> 107 fuzzy composition: 108 </p> 109 110 <p> 111 \(f_{R \circ S} (\langle x,z\rangle ) = \max_{y \in Y} \min(f_R (\langle x,y\rangle ), f_S (\langle y,z\rangle ))\) 112 </p> 113 114 <p> 115 hands-on example: 116 </p> 117 118 <p> 119 <img src="img/fuzzy-composition.png" alt="Fuzzy composition table" /> 120 </p> 121 122 <div id="Probability and Uncertainty-Vagueness: Fuzzy Set Theory-Evaluation"><h3 id="Evaluation">Evaluation</h3></div> 123 <ul> 124 <li> 125 good -- flexible, coincides with classical set theory, sever successful applications of fuzzy control 126 127 <li> 128 bad -- requires many arbitrary choices, tends to blur differences between probabilistic uncertainty/ambiguity/vagueness 129 130 </ul> 131 132 <div id="Probability and Uncertainty-Uncertainties: Probability Theory"><h2 id="Uncertainties: Probability Theory">Uncertainties: Probability Theory</h2></div> 133 <div id="Probability and Uncertainty-Uncertainties: Probability Theory-General"><h3 id="General">General</h3></div> 134 <p> 135 main interpretations of probability theory: 136 </p> 137 <ul> 138 <li> 139 optivist (frequentist) probability 140 141 <ul> 142 <li> 143 frequentism: probability is only property of repeated experiments 144 145 <li> 146 probability of event: limit of <em>relative frequency</em> of occurrence of event, as number of repetitions goes to infinity 147 148 </ul> 149 <li> 150 subjective probability 151 152 <ul> 153 <li> 154 Bayesianism: probability is an expression of our uncertainty and beliefs 155 156 <li> 157 probability of event: <em>degree of belief</em> of idealized rational individual 158 159 </ul> 160 </ul> 161 162 <p> 163 sample space Ω: set of single outcomes of experiment 164 </p> 165 166 <p> 167 event space E: things that have probability (subsets of sample space). if sample space is finite, event space is usually power set. 168 </p> 169 170 <div id="Probability and Uncertainty-Uncertainties: Probability Theory-Axioms of probability"><h3 id="Axioms of probability">Axioms of probability</h3></div> 171 <p> 172 for any event A, B: 173 </p> 174 <ul> 175 <li> 176 \(0 \leq P(A) \leq 1\) 177 178 <li> 179 \(P(\Omega) = 1\) 180 181 <li> 182 \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\) 183 184 </ul> 185 186 <p> 187 can derive: 188 </p> 189 <ul> 190 <li> 191 \(P({}) = 0\) 192 193 <li> 194 \(P(\Omega) = 1\) 195 196 <li> 197 \(\text{if} A \subset B, P(A) \leq P(B)\) 198 199 </ul> 200 201 <p> 202 conditional probability ("A given B"): 203 </p> 204 205 <p> 206 \(P(A|B) = \frac{P(A \cap B)}{P(B)}\) 207 </p> 208 209 <div id="Probability and Uncertainty-Uncertainties: Probability Theory-Joint probability distributions"><h3 id="Joint probability distributions">Joint probability distributions</h3></div> 210 <p> 211 for a set of random variables, it gives probability of every atomic even on those random variables. 212 </p> 213 214 <p> 215 e.g. P(Toothache, Catch, Cavity): 216 </p> 217 218 <table> 219 <tr> 220 <th> 221 222 </th> 223 <th colspan="2"> 224 toothache 225 </th> 226 <th colspan="2"> 227 ¬ toothache 228 </th> 229 </tr> 230 <tr> 231 <td> 232 233 </td> 234 <td> 235 catch 236 </td> 237 <td> 238 ¬ catch 239 </td> 240 <td> 241 catch 242 </td> 243 <td> 244 ¬ catch 245 </td> 246 </tr> 247 <tr> 248 <td> 249 cavity 250 </td> 251 <td> 252 0.108 253 </td> 254 <td> 255 0.012 256 </td> 257 <td> 258 0.072 259 </td> 260 <td> 261 0.008 262 </td> 263 </tr> 264 <tr> 265 <td> 266 ¬ cavity 267 </td> 268 <td> 269 0.015 270 </td> 271 <td> 272 0.064 273 </td> 274 <td> 275 0.144 276 </td> 277 <td> 278 0.576 279 </td> 280 </tr> 281 </table> 282 283 <p> 284 inference by enumeration: 285 </p> 286 <ul> 287 <li> 288 for any proposition Φ, sum atomic events where it's true -- \(P(\Phi) = \sum_{\omega : \omega \models \Phi} P(\omega)\) 289 290 <li> 291 compute conditional probability by selecting cells -- e.g. for P(¬ cavity | toothache), select toothache column and (¬ cavity) cells 292 293 </ul> 294 295 <p> 296 use Bayes' rule for opposite conditionals (like finding P(disease | symptom) from P(symptom | disease)) 297 </p> 298 299 <div id="Probability and Uncertainty-Uncertainties: Probability Theory-Bayesian networks"><h3 id="Bayesian networks">Bayesian networks</h3></div> 300 <p> 301 simple graphical notation for 302 </p> 303 <ul> 304 <li> 305 conditional independence assertions 306 307 <li> 308 compact specification of full joint distributions 309 310 </ul> 311 312 <p> 313 syntax: 314 </p> 315 <ul> 316 <li> 317 set of nodes, one per variable 318 319 <li> 320 directed acyclic graph, with a link meaning "directly influences" 321 322 <li> 323 conditional distribution for each node given its parents -- \(P(X_i | Parents(X_i))\) 324 325 </ul> 326 327 <p> 328 topology example: 329 </p> 330 331 <p> 332 <img src="img/bayesian-topology.png" alt="Bayesian network topology" /> 333 </p> 334 335 <p> 336 what does it mean? 337 </p> 338 <ul> 339 <li> 340 <em>weather</em> is independent of other variables 341 342 <li> 343 <em>toothache</em> and <em>catch</em> are conditionally independent given <em>cavity</em>. 344 345 </ul> 346 347 <div id="Probability and Uncertainty-Uncertainties: Probability Theory-Evaluation of probabilities"><h3 id="Evaluation of probabilities">Evaluation of probabilities</h3></div> 348 <ul> 349 <li> 350 good -- sound theoretical basis, can be extended to decision-making, some good tools available 351 352 <li> 353 bad -- not always computationally easy, need lots of data which may be hard to get 354 355 </ul> 356 357 </body> 358 </html>