Theory of Random Variable and Noise

If the ultimate receiver knew in advance the message output from the originating source, there would be no need to communicate; and if the propagation of electromagnetic signals were not disturbed by nature, to communicate the message would be no problem. The word “random” means “unpredictable”, on the basis of what we know about the past of a phenomenon, we are unable to predict its future in detail.

Randomness in the Real World

Our inability to predict the detailed future of a random phenomenon may arise either from ignorance or laziness: to the limit of our knowledge, the laws governing a progression of events may be fundamentally random (as in quantum physics); or they may be so complicated and involve such critical dependence on initial conditions (as in coin tossing) that we deem it unprofitable to undertake a detailed analysis.

A certain example of randomness in communication system is the transmission of radio waves through the ionosphere. The radio waves at certain frequencies are refracted as they pass through the ionized gas that constitutes
the ionosphere. The degree of refraction depends on the detailed structure of the ionosphere, which depends, in turn, on the amount of ionizing solar radiation, the incidence and velocities of meteors, and on many other factors.

The voltage at the terminals of the receiving antenna is the resultant of a number of waves travelling over a variety of different paths.
Although we cannot predict exactly what the antenna output voltage will be, we find experimentally that certain average properties do exhibit a reasonable regularity.
This statistical regularity of averages is an experimentally verifiable phenomenon in many different situations involving randomly varying quantities and this is the domain of the mathematical field of probability and statistics.

Random Experiments

The term experiment is used in probability theory to describe a process whose outcome can not be fully predicted because the conditions under which it is performed can not be predetermined with sufficient accuracy and completeness.
Tossing a coin, rolling a die, and drawing a card from a deck are some example of such experiments.

Remember

An experiment is called random when the conditions of the measurement are not predetermined with sufficient accuracy and completeness to permit a precise prediction of the result of a trial. Whether an experiment should be considered random depends on the precision with which we wish to distinguish between possible outcomes.

Mathematical model of probability theory

Mathematical models prove useful in predicting the results of experiments in the real world when two conditions are met:
First, the physical entities and their properties must be reflected in the model.
Second, the properties of the model must be mathematically consistent and permissive of analysis.

• The real-world random experiments involve three pertinent entities:

1. The set of all possible experimental outcomes.
2. The grouping of these outcomes into classes, called results, between which we wish to distinguish.
3. The relative frequencies with which these classes occur in a long sequence of independent trials of the experiment.

• In the mathematical model of probability theory the corresponding abstractions are called:

1. The sample space.
2. The set of events.
3. The probability measure defined on these events.

Fundamental Definitions

Sample space: Sample space is a collection of all possible and separately identifiable outcomes of an experiment.

Event: A set of sample points. We usually label events by capital letters, such as A, B, ….., or A₁, A₂, …..
An event is concisely defined by the expression.

A = {ω: some condition on ω is satisfied}

Probability measure
An assignment of real numbers to the events defined on sample space. The probability of an event A is denoted P[A]. The conditions that the assignment must satisfy will be discussed subsequently.

Definitions

The definition of a sample space Ω and events such as A, B, … implies the existence of certain other identifiable sets of points.

Complement:

The complement of A, denoted A^C, is the event containing all points in Ω but not in A.

Properties

In a long sequence of N independent trials of a random experiment in the real world the results {Ai} and the observed frequencies {f_N(A_i)} with which these results occur meet certain conditions:

1. The relative frequency f_N(A_i) of every result satisfies the inequalities 0 ≤ f_N(A_i) ≤ 1.
2. Every trial of an experiment has an outcome.
3. If two results A and B are mutually exclusive, then f_N(Aor B) = f_N(A) + f_N(B).

Or in other words,

(i) To every event A_i, a unique number P[A_i] is assigned such that 0 ≤ P (A_i) ≤ 1.

(ii) P(Ω) = 1.

(iii) If AB = ∅, P[A ∪ B] = P[A] + P[B].