probability-uncertainty.wiki (4501B)
1 = Probability and Uncertainty = 2 one often has to deal with info that is underspecified, incomplete, vague, etc. 3 4 logic by itself is not sufficient for these problems. 5 6 == Vagueness: Fuzzy Set Theory == 7 model theory often based on set theory 8 9 fuzzy set theory allows something to be _to some degree_ an element of a set 10 11 dominant approach to vagueness (mostly because wtf else can you do) 12 13 === Fuzzy sets === 14 * universe U, object x ∈ U 15 * membership function for fuzzy set A is defined to be function $f_A$ from U to [0,1]: 16 * $f_A(x)=y$: x is a member of A to degree y 17 * $f_A(x)=1$: x is certainly member of A 18 * $f_A(x)=0$: x is certainly not a member of A 19 * ${x | f_A(x)>0}$: support of A 20 21 modifiers (hedges) 22 23 {{file:img/modifiers-hedges.png|Example of modifiers' effects on graphs}} 24 25 operations on fuzzy sets: 26 * complement: $f_{\sim{A}}(x)=1-f_A(x)$ 27 * union: $f_{A\cup B}(x)=\max(f_A(x),f_B(x))$ 28 * intersection: $f_{A\cap B}(x)=\min(f_A(x), f_B(x))$ 29 * subset: $A\subset B \iff \forall{x}(f_A(x)\leq f_B(x))$ 30 31 semantics -- multivalued fuzzy logic 32 * v(¬ A) = 1-v(A) 33 * v(A ∨ B) = max(v(A), v(B)) 34 * v(A ∧ B) = min(v(A), v(B)) 35 * v(A → B) = min(1, 1 - v(A) + v(B)) 36 37 === Fuzzy relations === 38 fuzzy sets can denote fuzzy relations between objects. e.g. approximately equal, close to, much larger than, etc. 39 40 fuzzy composition: 41 42 $f_{R \circ S} (\langle x,z\rangle ) = \max_{y \in Y} \min(f_R (\langle x,y\rangle ), f_S (\langle y,z\rangle ))$ 43 44 hands-on example: 45 46 {{file:img/fuzzy-composition.png|Fuzzy composition table}} 47 48 === Evaluation === 49 * good -- flexible, coincides with classical set theory, sever successful applications of fuzzy control 50 * bad -- requires many arbitrary choices, tends to blur differences between probabilistic uncertainty/ambiguity/vagueness 51 52 == Uncertainties: Probability Theory == 53 === General === 54 main interpretations of probability theory: 55 * optivist (frequentist) probability 56 * frequentism: probability is only property of repeated experiments 57 * probability of event: limit of _relative frequency_ of occurrence of event, as number of repetitions goes to infinity 58 * subjective probability 59 * Bayesianism: probability is an expression of our uncertainty and beliefs 60 * probability of event: _degree of belief_ of idealized rational individual 61 62 sample space Ω: set of single outcomes of experiment 63 64 event space E: things that have probability (subsets of sample space). if sample space is finite, event space is usually power set. 65 66 === Axioms of probability === 67 for any event A, B: 68 * $0 \leq P(A) \leq 1$ 69 * $P(\Omega) = 1$ 70 * $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ 71 72 can derive: 73 * $P({}) = 0$ 74 * $P(\Omega) = 1$ 75 * $\text{if} A \subset B, P(A) \leq P(B)$ 76 77 conditional probability ("A given B"): 78 79 $P(A|B) = \frac{P(A \cap B)}{P(B)}$ 80 81 === Joint probability distributions === 82 for a set of random variables, it gives probability of every atomic even on those random variables. 83 84 e.g. P(Toothache, Catch, Cavity): 85 86 | | toothache | > | ¬ toothache | > | 87 |----------|-----------|---------|-------------|---------| 88 | | catch | ¬ catch | catch | ¬ catch | 89 | cavity | 0.108 | 0.012 | 0.072 | 0.008 | 90 | ¬ cavity | 0.015 | 0.064 | 0.144 | 0.576 | 91 92 inference by enumeration: 93 * for any proposition Φ, sum atomic events where it's true -- $P(\Phi) = \sum_{\omega : \omega \models \Phi} P(\omega)$ 94 * compute conditional probability by selecting cells -- e.g. for P(¬ cavity | toothache), select toothache column and (¬ cavity) cells 95 96 use Bayes' rule for opposite conditionals (like finding P(disease | symptom) from P(symptom | disease)) 97 98 === Bayesian networks === 99 simple graphical notation for 100 * conditional independence assertions 101 * compact specification of full joint distributions 102 103 syntax: 104 * set of nodes, one per variable 105 * directed acyclic graph, with a link meaning "directly influences" 106 * conditional distribution for each node given its parents -- $P(X_i | Parents(X_i))$ 107 108 topology example: 109 110 {{file:img/bayesian-topology.png|Bayesian network topology}} 111 112 what does it mean? 113 * _weather_ is independent of other variables 114 * _toothache_ and _catch_ are conditionally independent given _cavity_. 115 116 === Evaluation of probabilities === 117 * good -- sound theoretical basis, can be extended to decision-making, some good tools available 118 * bad -- not always computationally easy, need lots of data which may be hard to get 119 120