I recently grew a mild obsession to understand complex numbers. It all started while working with Tirza Routenberg and Neta Zimerman on the analysis of seismic array data. The array processing community will usually model measurements as complex numbers. This approach is so natural in the signal processing community, that the canonical reference for array processing, [vanTrees2002], never even stops to explain why? Being the statistician that I am, it puzzled me: the measurements are the instantanous compression of soil, why would you want to represent that with a complex number?

The following is my current understanding for the reasons that the signal processing, and array processing communities, will model measurements as complex numbers. The tl;dr version is that: (a) The merit of complex numbers is due to their representation via complex exponentials. Shifting a complex valued sine-wave in time, is merely multiplying its complex exponential representation by some (complex) constant. (b) Any real-valued function/signal, may be mapped to its baseband representation, a.k.a., its complex envelope, without loss of information. (c) These mappings/representations are useful since followup processing will typically include linear systems (convolutions), deconvolutions, Fourier transforms, etc, which are easier, both computationally and analytically, when operating with complex exponentials.

Now with details.

Useful for Computations

The usual argument one receives when asking a physicist or an electrical engineer “why complex” is:
(a) Some measurements are complex.
(b) Super useful for handling waves.
(c) Super useful for linear systems.
Let’s parse them one by one.

Some measurements are complex

Some measurements, by their nature, respect the arithmetic of complex numbers. This is the case when measuring current and voltage. This is not the case in acoustics/seismology, where measurements represent the compression of air/soil.

Super useful for waves

This is an important point which may be initially unclear to someone, like myself, who never really understood the difference between a wave, and any other function of time. A wave is a function of time and space, but it is not an arbitrary function. It represents a disturbance that propagates in time and space, so adjacent values are interconnected. It is not only smooth, but also has to satisfy the wave equation. Without going in the details of partial differential equations I will just say that a sine wave satisfies the wave equation, and thus any solution, i.e., any wave, is usually recovered by presenting the solution as scaled and shifted sine waves. Sine waves and their shifts are best represented with complex numbers, as I will soon demonstrate. For a full explanation I recommend [Smith2002].

Super useful for convolutions

Say you are analyzing the effect of a linear time invariant system (LTI), aka a convolution. It is a well known fact that Fourier diagonalizes the convolution. Put differently, the spectrum of the output of an LTI, is a point-wise multiplication with some other function. We will show this later in this post.

Sensor Arrays

As the name suggests, the field of sensor arrays deals with measurements from, well, sensor arrays. It turns out that when analyzing data from an array of sensors, complex numbers soon arise. Why is this? Consider the real-valued measurement, \(f_k(t)\) of sensor \(k\) at time \(t\). Sensor \(k\) and \(k'\) measure the same function at different locations. Because this is the same function, measurements differ in their temporal lag: \(f_{k'}(t)=f_k(t-\tau_k)\). Now enter a crucial fact about sine waveforms. Say \(f(t)\) is a sine wave in the complex plane: \(f(t)=\cos(t) + i \sin(t)\) where \(i:=\sqrt{-1}\). In complex exponential notation this is \(f(t)=e^{i t}\). Now presenting the time shift in complex exponential notation: \(f(t-\tau)=\sqrt{\cos^2(t)+\sin^2(t)} e^{i (t+\tau)}=e^{it} e^{i\tau}=f(t)e^{i\tau}\). This is why we say that a shift in time is a multiplication in frequency.
For some intuition, imagine that \(f(t)\) is the helix around a screw. To shift time, i.e. evaluate \(f(t)\) at \(t-\tau\) one can either look at position \(t-\tau\), or keep looking at position \(t\), but advance the screw a distance of \(\tau\). The “advancing of the screw” is the effect of the complex multiplication.

Since our sensor array measures time-shifted versions of the same signal, \(\{f(t-\tau_k)\}_k\), it would be nice if \(f(t)\) could be decomposed as a linear combination of complex exponentials. But this is what the real-to-complex Fourier transform does! Now since the first thing that will be done to array measurements is Fourier transforming them, why not start with directly there? This is known, at least in [vanTrees2002], as the frequency domain snapshot model.

Signal Processing

The previous discussion implies that for array signal processing, and given some assumptions that we skipped, one should adopt the frequency domain snapshot model. But if you ever practiced signal processing, you may know that the time domain snapshot model is no less popular, not only for arrays, and will often use complex numbers. So again, why model measurements as complex numbers? In particular, when no arrays are involved?

One reason has to do with the fact that the term signal processing includes both digital communication, and data analysis.

In digital communication, it is quite common that one needs to transmit a message over a analogue channel (e.g. radio). Because the channel is analogue, it essentially transmits waves. The message to be transmitted has to be encoded by shifting and scaling wave functions. This practice is known as modulation, and it is best done using complex exponentials because it, again, involves shifting and scaling wave functions. Is this also the case for data analysis?

When signal processing for data analysis, there is no transmission, only reception. One may argue that in data analyses we are “decripting nature’s messages”, but this romantic view has its limitation: we do not know the encoding mechanism used by nature, and the task is not decoding.

So why model measurements as complex numbers? My answer to this is the complex envelope. In my view, the matter is best described in [Schreier2010], and the argument is essentially, that there is nothing to lose. The complex envelope is also known under the more informative name of equivalent baseband signal. It is essentially a representation of the real-value signal, using a minimal spectrum. Minimal in the sense that negative frequencies are canceled, and the remaining are shifted to some origin. The price to pay for this “spectral compactness”, is that the signal is no longer real valued. One can always convert from the complex envelope to the real-valued signal, and vice-versa.

The baseband noise

I may have convinced you, and myself, that the complex envelope loses nothing, and may facilitate further processing which is easier with complex numbers. This is true, but an important detail to mind when adopting this time domain complex envelope snapshot model, is the noise. In the real-valued time domain, we usually use a white, Gaussian noise process, to model noise. But what is the complex envelope of a Gaussian white noise process? Is it Gaussian? It is white? The answer may be found in [vanTrees2002] circa Eq(5.79), or more rigorously in [Viswanathan2006]. The answer is approximately affirmative, meaning that one may use a white (proper) Gaussian process as the complex envelope of the real-valued noise process.

Complex Sinusoids Diagonalize the Convolution

Complex sinusoids diagonalize the convolution. Put differently: complex sinusoids are eigenfunctions of any linear time invariant system (LTI). We said this earlier informally, but now we can be more formal about it.

Consider the operation of an LTI, \(\mathcal{H}\) on a complex sinusoid, \(f(t)=e^{i \omega t}\). Denote \(g:=\mathcal{H}\{f\}\), \(f^*(t)=f(t-\tau)\), \(g^*(t)=g(t-\tau)\). By definition \(g^*=\mathcal{H}\{f^*\}=\mathcal{H}\{f(t-\tau)\}=\mathcal{H}\{e^{-i\omega\tau}f(t)\}\). By linearity of \(\mathcal{H}\): \(\mathcal{H}\{e^{-i\omega\tau}f(t)\}=e^{-i\omega\tau}\mathcal{H}\{f(t)\}=e^{-i\omega\tau}g\). Now because of time-invariance \(e^{-i\omega\tau}g=g^*\), which is satisfied if \(g\) is a complex sinusoid.

Conclusions

1. For array processing, where time-shifting is key, a frequency-domain-snapshot-model is a natural approach.

2. For general analysis of real-valued signal, complex modeling may be less obvious but brings benefits. Thinking of the complex-envelope of a real-valued signal is harmless, provided you take care of the right noise model.

Acknowledgements

I am thankful for the fruitful conversations on the matter with Tirza Routnerberg, Jont Allen, Roy Lederman, and Armin Schwartzman.

References

[Smith2002] Steven Smith, Digital Signal Processing: A Practical Guide for Engineers and Scientists, 1st edition (Amsterdam; Boston: Newnes, 2002).

[Schreier2010] Peter J. Schreier and Louis L. Scharf, Statistical Signal Processing of Complex-Valued Data: The Theory of Improper and Noncircular Signals (Cambridge: Cambridge University Press, 2010).

[vanTrees2002] Harry L. Van Trees, Optimum Array Processing: Part IV of Detection, Estimation, and Modulation Theory, 1st edition (New York: Wiley-Interscience, 2002).

[Viswanathan2006] R. Viswanathan, “On the Autocorrelation of Complex Envelope of White Noise,” IEEE Transactions on Information Theory 52, no. 9 (September 2006): 4298–99.