index.md (4507B)
1 +++ 2 title = "Probability and Uncertainty" 3 template = 'page-math.html' 4 +++ 5 # Probability and Uncertainty 6 one often has to deal with info that is underspecified, incomplete, vague, etc. 7 8 logic by itself is not sufficient for these problems. 9 10 ## Vagueness: Fuzzy Set Theory 11 model theory often based on set theory 12 13 fuzzy set theory allows something to be _to some degree_ an element of a set 14 15 dominant approach to vagueness (mostly because wtf else can you do) 16 17 ### Fuzzy sets 18 * universe U, object x ∈ U 19 * membership function for fuzzy set A is defined to be function $f_A$ from U to [0,1]: 20 * $f_A(x)=y$: x is a member of A to degree y 21 * $f_A(x)=1$: x is certainly member of A 22 * $f_A(x)=0$: x is certainly not a member of A 23 * ${x | f_A(x)>0}$: support of A 24 25 modifiers (hedges) 26 27  28 29 operations on fuzzy sets: 30 * complement: $f_{\sim{A}}(x)=1-f_A(x)$ 31 * union: $f_{A\cup B}(x)=\max(f_A(x),f_B(x))$ 32 * intersection: $f_{A\cap B}(x)=\min(f_A(x), f_B(x))$ 33 * subset: $A\subset B \iff \forall{x}(f_A(x)\leq f_B(x))$ 34 35 semantics -- multivalued fuzzy logic 36 * v(¬ A) = 1-v(A) 37 * v(A ∨ B) = max(v(A), v(B)) 38 * v(A ∧ B) = min(v(A), v(B)) 39 * v(A → B) = min(1, 1 - v(A) + v(B)) 40 41 ### Fuzzy relations 42 fuzzy sets can denote fuzzy relations between objects. e.g. approximately equal, close to, much larger than, etc. 43 44 fuzzy composition: 45 46 $f_{R \circ S} (\langle x,z\rangle ) = \max_{y \in Y} \min(f_R (\langle x,y\rangle ), f_S (\langle y,z\rangle ))$ 47 48 hands-on example: 49 50  51 52 ### Evaluation 53 * good -- flexible, coincides with classical set theory, sever successful applications of fuzzy control 54 * bad -- requires many arbitrary choices, tends to blur differences between probabilistic uncertainty/ambiguity/vagueness 55 56 ## Uncertainties: Probability Theory 57 ### General 58 main interpretations of probability theory: 59 * optivist (frequentist) probability 60 * frequentism: probability is only property of repeated experiments 61 * probability of event: limit of _relative frequency_ of occurrence of event, as number of repetitions goes to infinity 62 * subjective probability 63 * Bayesianism: probability is an expression of our uncertainty and beliefs 64 * probability of event: _degree of belief_ of idealized rational individual 65 66 sample space Ω: set of single outcomes of experiment 67 68 event space E: things that have probability (subsets of sample space). if sample space is finite, event space is usually power set. 69 70 ### Axioms of probability 71 for any event A, B: 72 * $0 \leq P(A) \leq 1$ 73 * $P(\Omega) = 1$ 74 * $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ 75 76 can derive: 77 * $P({}) = 0$ 78 * $P(\Omega) = 1$ 79 * $\text{if} A \subset B, P(A) \leq P(B)$ 80 81 conditional probability ("A given B"): 82 83 $P(A|B) = \frac{P(A \cap B)}{P(B)}$ 84 85 ### Joint probability distributions 86 for a set of random variables, it gives probability of every atomic even on those random variables. 87 88 e.g. P(Toothache, Catch, Cavity): 89 90 | | toothache | > | ¬ toothache | > | 91 |----------|-----------|---------|-------------|---------| 92 | | catch | ¬ catch | catch | ¬ catch | 93 | cavity | 0.108 | 0.012 | 0.072 | 0.008 | 94 | ¬ cavity | 0.015 | 0.064 | 0.144 | 0.576 | 95 96 inference by enumeration: 97 * for any proposition Φ, sum atomic events where it's true -- $P(\Phi) = \sum_{\omega : \omega \models \Phi} P(\omega)$ 98 * compute conditional probability by selecting cells -- e.g. for P(¬ cavity | toothache), select toothache column and (¬ cavity) cells 99 100 use Bayes' rule for opposite conditionals (like finding P(disease | symptom) from P(symptom | disease)) 101 102 ### Bayesian networks 103 simple graphical notation for 104 * conditional independence assertions 105 * compact specification of full joint distributions 106 107 syntax: 108 * set of nodes, one per variable 109 * directed acyclic graph, with a link meaning "directly influences" 110 * conditional distribution for each node given its parents -- $P(X_i | Parents(X_i))$ 111 112 topology example: 113 114  115 116 what does it mean? 117 * _weather_ is independent of other variables 118 * _toothache_ and _catch_ are conditionally independent given _cavity_. 119 120 ### Evaluation of probabilities 121 * good -- sound theoretical basis, can be extended to decision-making, some good tools available 122 * bad -- not always computationally easy, need lots of data which may be hard to get 123 124