Random Variables and Random Processes Some basics of probability theory are discussed before going to random variables.
Basics of Probability Theory Probability of an event A represented by P (A) is given by
P (A) =
where, NS is the number of times the experiment is carried and NA is number of times the event A occured. Probability of any event can be exactly caluculated only when the number of experiments is huge ideally infinity. Hence, we generally go for relative probability which is given above. Clearly, 0 ≤ P (A) ≤ 1. Let 0 S 0 be the sample space having N events A1 , A2 , A3 , · · · AN . Two events are said to be mutually exclusive or statistically independent if Ai ∩ Aj = φ and SN i=1 Ai = S for all i and j.
Joint probability of two events A and B which is defined as the probability of the occurence of both the events A and B is given by
P (A ∩ B) =
Conditional Probability Conditional probability of two events A and B represented as P (A/B) and defined as the probability of the occurence of event A after the occurence of B is given by 1
P (A/B) = P (A ∩ B)/P (B) P(B/A) = P(A ∩B)/P (A) =⇒ P (A/B).P (B) = P (B/A).P (A) = P (A ∩ B) Chain rule Let us consider a chain of events A1 , A2 , A3 , · · · AN which are dependent on each other. Then the probability of occurence of the sequence
P (AN , AN −1 , AN −2 , · · · A1 ) = P (AN /AN −1 , AN −2 , · · · A1 ).P (AN −1 /AN −2 , AN −3 , · · · A1 ) · · · P (A2 /A1 ).P
Bayes Rule A2
Figure 1: Partition space In the above figure, if A1 , A2 , A3 , · · · A5 partition the sample space S, then A1 ∩ B, A2 ∩ B, A3 ∩ B, A4 ∩ B, andA5 ∩ B partition B. therefore,
P (B) = =
i=1 n X
P (Ai ∩ B) P (B/Ai ).P (Ai )
P (Ai /B) = P (Ai ∩ B)/P (B) =
P (B/Ai ).P (Ai ) n X P (B/Ai ).P (Ai ) i=1
In the above equation, P (Ai /B) is called posterior probability, P (B/Ai ) is called n X likelihood, P (Ai ) is called prior probability and P (B/Ai ).P (Ai ) is called i=1
Random Variable Random variable is a function whose domain is sample space and whose range is the set of real numbers.
Probabilistic description of a Random Variable Cummulative Probability Distribution: It is represented as FX (x) and defined as
FX (x) = P (X ≤ x) If x1 < x2 then FX (x1 ) < FX (x2 ) and 0 ≤ FX (x) ≤ 1. Probability Density Function: It is represented as fX (x) and defined as
d FX (x) dx Z x2 =⇒ P (x1 ≤ X ≤ x2 ) = fX (x) dx fX (x) =
Basics of Signals and Systems Feb 1st, 2007