Basics of Signals and Systems Feb 1st, 2007
Random Variables and Random Processes Some basics of probability theory are discussed before going to random variables.
Basics of Probability Theory Probability of an event A represented by P (A) is given by
P (A) =
NA NS
where, NS is the number of times the experiment is carried and NA is number of times the event A occured. Probability of any event can be exactly caluculated only when the number of experiments is huge ideally infinity. Hence, we generally go for relative probability which is given above. Clearly, 0 ≤ P (A) ≤ 1. Let 0 S 0 be the sample space having N events A1 , A2 , A3 , · · · AN . Two events are said to be mutually exclusive or statistically independent if Ai ∩ Aj = φ and SN i=1 Ai = S for all i and j.
Joint Probability:
Joint probability of two events A and B which is defined as the probability of the occurence of both the events A and B is given by
P (A ∩ B) =
NA∩B NS
Conditional Probability Conditional probability of two events A and B represented as P (A/B) and defined as the probability of the occurence of event A after the occurence of B is given by 1
P (A/B) = P (A ∩ B)/P (B) P(B/A) = P(A ∩B)/P (A) =⇒ P (A/B).P (B) = P (B/A).P (A) = P (A ∩ B) Chain rule Let us consider a chain of events A1 , A2 , A3 , · · · AN which are dependent on each other. Then the probability of occurence of the sequence
P (AN , AN −1 , AN −2 , · · · A1 ) = P (AN /AN −1 , AN −2 , · · · A1 ).P (AN −1 /AN −2 , AN −3 , · · · A1 ) · · · P (A2 /A1 ).P
Bayes Rule A2
A1
A5
B
A4
A3
Figure 1: Partition space In the above figure, if A1 , A2 , A3 , · · · A5 partition the sample space S, then A1 ∩ B, A2 ∩ B, A3 ∩ B, A4 ∩ B, andA5 ∩ B partition B. therefore,
P (B) = =
n X
i=1 n X
P (Ai ∩ B) P (B/Ai ).P (Ai )
i=1
P (Ai /B) = P (Ai ∩ B)/P (B) =
P (B/Ai ).P (Ai ) n X P (B/Ai ).P (Ai ) i=1
2
In the above equation, P (Ai /B) is called posterior probability, P (B/Ai ) is called n X likelihood, P (Ai ) is called prior probability and P (B/Ai ).P (Ai ) is called i=1
evidence.
Random Variable Random variable is a function whose domain is sample space and whose range is the set of real numbers.
Probabilistic description of a Random Variable Cummulative Probability Distribution: It is represented as FX (x) and defined as
FX (x) = P (X ≤ x) If x1 < x2 then FX (x1 ) < FX (x2 ) and 0 ≤ FX (x) ≤ 1. Probability Density Function: It is represented as fX (x) and defined as
d FX (x) dx Z x2 =⇒ P (x1 ≤ X ≤ x2 ) = fX (x) dx fX (x) =
x1
3
