Introduction

Frequency Shift Keying (FSK) remains a foundational modulation scheme in wireless communications, prized for its resilience to amplitude noise and straightforward implementation. In cognitive radio (CR) systems—where unlicensed secondary users must dynamically access licensed spectrum without harming primary users—FSK’s spectral efficiency becomes a key design metric. This article explores the factors that influence FSK spectral efficiency in CR environments, analyzes trade-offs, and reviews strategies to maximize throughput while maintaining rigorous interference constraints.

Fundamentals of FSK in Wireless Communications

FSK encodes digital data by shifting the carrier frequency among a predefined set of discrete frequencies. For binary FSK (BFSK), two frequencies represent 0 and 1. In M-ary FSK (M = 4, 8, 16, etc.), each symbol carries log2(M) bits. The minimum spacing between adjacent frequencies for orthogonal FSK is 1/Ts, where Ts is the symbol period, leading to a total bandwidth roughly equal to M × (1/Ts). This bandwidth linearity with M sets FSK apart from phase- or amplitude-based modulations and strongly influences its spectral efficiency in multichannel environments.

The classical spectral efficiency (SE) formula for any modulation is SE = Rb / B, where Rb is the bit rate and B is the occupied bandwidth. For non-coherent orthogonal FSK, the upper bound is often expressed as SE ≤ (log2 M) / (M × (1 + β)), where β accounts for any guard bands or pulse shaping. This bound reveals a fundamental trade-off: doubling M increases the numerator log2 M slowly while the denominator M grows linearly, causing SE to peak at small M (typically M = 2 or 4) and then decay. In practice, adaptive techniques can modify this static picture.

Cognitive Radio Context and Dynamic Spectrum Access

Cognitive radio is built on dynamic spectrum access (DSA), where secondary users sense the spectrum, identify vacant “white spaces,” and occupy them without interfering with licensed primary users. The Federal Communications Commission (FCC) and global regulators have opened television white spaces and other bands for unlicensed DSA, spurring research into modulations that can adapt quickly to changing channel occupancy. FSK, with its constant envelope and robustness to nonlinear amplification, is attractive for low-cost transceivers that must vacate channels on short notice.

In a CR network, the secondary user (SU) must achieve reliable communication within a narrow time frame before the primary user (PU) may reclaim the channel. Spectral efficiency for the SU is not just raw bps/Hz; it must account for the overhead of spectrum sensing (Yucek & Arslan, 2009), the probability of false alarms, and the need to keep out-of-band emissions low. FSK’s ability to implement non-coherent detection—requiring no phase recovery—reduces sensing latency and simplifies the receiver, but its poor SE relative to QAM remains a persistent challenge.

Factors Affecting Spectral Efficiency of FSK in CR Systems

Bandwidth Utilization and Modulation Order

Bandwidth occupancy for M-ary orthogonal FSK scales directly with M. For a fixed symbol rate, doubling M doubles the bandwidth, while the bit rate increases only by log2 M. For instance, BFSK (M=2) yields SE ≈ 1 bps/Hz under ideal orthogonal spacing, while 4-FSK yields SE ≈ 2/3 bps/Hz, assuming no guard bands. This inherent inefficiency is why high-order QAM (e.g., 64-QAM) often replaces FSK in high-throughput scenarios. However, in CR, the agility of FSK and its lower out-of-band emission can offset its SE penalty when the available white spaces are narrow or fragmented.

Interference Constraints and Primary User Protection

CR systems must guarantee that the interference temperature at the primary receiver remains below a threshold. For FSK, increased bandwidth increases the probability of overlapping with a PU channel, particularly if the SU’s frequency hops across multiple white spaces. Advanced spectrum sensing algorithms (e.g., energy detection, cyclostationary feature detection) must distinguish between FSK signals and noise with high accuracy. The detection probability and false alarm rate directly affect the effective throughput, as missed detections cause harmful interference and false alarms waste transmission opportunities.

Noise, Fading, and Channel Impairments

In Rayleigh or Nakagami fading channels, non-coherent FSK requires a higher signal-to-noise ratio (SNR) to achieve a given bit error rate (BER) compared to coherent modulations. This SNR requirement translates to a need for higher transmit power or lower coding rates, both of which reduce effective SE. For CR, the situation is compounded by PU interference: when a PU transmits on a nearby channel, the SU’s FSK signal must maintain a low power spectral density (PSD) to avoid raising the noise floor. Frequency-domain scheduling—allocating SUs to the least congested subbands—can mitigate this, but it adds complexity to the sensing and decision engine.

Spectrum Sensing Accuracy and Overhead

Every transmission in a CR system must be preceded by a sensing period. The longer the sensing duration, the more accurate the detection, but the less time remains for actual data communication. This trade-off is captured by the “sensing-throughput trade-off,” extensively studied for energy detection. For FSK, the overhead can be reduced by using the same FSK tones for both sensing and data—a technique known as simultaneous sensing and transmission. However, this approach increases the probability of self-interference and requires sophisticated cancellation circuits (Liang et al., 2015).

Strategies to Improve Spectral Efficiency of FSK in CR

Adaptive Modulation and Power Control

A cognitive radio can switch among BFSK, 4-FSK, and higher orders based on real-time SNR and PU activity. When the channel is clear and SNR high, the SU can use 4- or 8-FSK to increase data rate, accepting the bandwidth penalty. Conversely, during low SNR or when the PU is active nearby, the SU falls back to BFSK to minimize bandwidth and reduce interference risk. Power control must accompany modulation switching to ensure the SU’s PSD stays below the interference temperature limit set by the regulator. Such adaptive modulation algorithms typically rely on a lookup table mapping SINR to a modulation/coding scheme (MCS) and can dramatically improve average SE in time-varying white spaces.

Enhanced Spectrum Sensing Techniques

Improving the accuracy of spectrum sensing reduces the safety margins that protocols impose (e.g., requiring a 0 dB SNR margin in sensing). Cyclostationary detection exploits the periodic nature of FSK signals to distinguish them from noise even at very low SNR. Machine learning approaches, such as convolutional neural networks trained on FSK spectrograms, have shown promising detection probabilities above 0.99 at −10 dB SNR (Lee et al., 2019). Fewer false alarms mean the SU can transmit more often, increasing effective SE.

Multi-Carrier FSK (MC-FSK) and Non-Contiguous FSK

Rather than using a single FSK carrier that spans a contiguous block of spectrum, a CR can split its transmission across multiple narrowband FSK subcarriers placed in disjoint white spaces. This is conceptually similar to OFDM but with FSK modulation on each subcarrier. Non-contiguous FSK (NC-FSK) can achieve high aggregate SE by packing bits into all available tiny spectrum holes. The cost is increased complexity in the transmitter (multiple oscillators or fast frequency hopping) and receiver, as well as higher peak-to-average power ratio (PAPR). Nevertheless, for CR scenarios where white spaces are fragmented (e.g., TV whitespaces after digital switchover), NC-FSK can outperform continuous QAM in terms of total bits delivered per second over the entire available band.

Cooperative Relaying and Network Coordination

If multiple SUs cooperate to relay each other’s data, the effective SE of FSK can be improved through diversity gains. For example, a secondary node that senses a better channel can act as a relay for its neighbors, reducing the probability of outage. Cooperative spectrum sensing also improves detection accuracy, which indirectly boosts SE by allowing more aggressive modulation use. However, the overhead of relay control messages and synchronization—often requiring its own dedicated channel—must be subtracted from the gross SE.

Challenges and Trade-offs in Practical Deployment

Out-of-Band Emissions and Regulatory Limits

FSK’s constant envelope helps keep spectral regrowth low, but the sudden frequency transitions inherent in M-ary FSK produce sidelobes that can interfere with adjacent PU channels. Root-raised cosine pulse shaping or Gaussian filtering (GFSK) reduces these sidelobes at the cost of intersymbol interference (ISI) and a slight bandwidth expansion. In CR, regulators such as the FCC mandate strict emission masks (e.g., −55 dB below peak in TV whitespace bands). Meeting these masks often forces the SU to operate with lower effective bandwidth or to insert guard tones, reducing SE.

Hardware Limitations and Phase Noise

For coherent FSK detection, the receiver must lock to the carrier frequency accurately. In low-cost CR nodes, phase noise from cheap oscillators can degrade symbol decisions, especially for higher order FSK where tone spacing narrows (because the total bandwidth is fixed, but subcarrier spacing decreases as M increases). Non-coherent detection avoids this issue but suffers a ~3 dB SNR penalty. The choice between coherent and non-coherent FSK in CR is therefore a trade-off: coherent offers better SE at higher cost and complexity, while non-coherent is simpler but less efficient.

Mobility and Doppler Effects

In mobile CR scenarios (e.g., vehicular networks), Doppler shifts stretch or compress the FSK tone spacing, potentially causing inter-tone interference. Adaptive frequency control that tracks the Doppler and adjusts the center frequency can mitigate this, but the algorithm consumes additional sensing time. Moreover, the coherence time of the channel may be shorter than the symbol period for high-order FSK, causing rapid changes in the received signal that further degrade SE.

Future Directions and Research Opportunities

The pursuit of higher spectral efficiency from FSK in CR continues. One promising avenue is combining FSK with index modulation (IM), where the set of activated tones carries additional information. In FM-FSK (frequency modulated FSK), the exact frequencies are varied subtly to encode extra bits, challenging the traditional orthogonality condition. Another direction exploits massive MIMO CR nodes: by beamforming the FSK signal toward the PU’s nulls, the interference constraint is relaxed, allowing higher transmit power and thus higher-order FSK.

Furthermore, machine-learning-driven cognitive engines that jointly optimize sensing duration, modulation order, and power allocation in real time are under active investigation. These systems can learn the statistical behavior of PUs and the channel, enabling preemptive adaptation that minimizes the sensing overhead. Combining FSK with full-duplex radio—where the SU transmits and senses simultaneously—could theoretically double the effective SE, though practical self-interference cancellation remains challenging.

Conclusion

Spectral efficiency of FSK in cognitive radio systems is shaped by a complex interplay of modulation parameters, sensing accuracy, interference constraints, and hardware capabilities. While FSK’s inherent SE is lower than QAM in static wideband channels, its agility, constant envelope, and compatibility with non-coherent detection make it a valuable tool for opportunistic access in fragmented and interference-prone spectrum. Adaptive modulation, advanced sensing, multi-carrier techniques, and cooperative relaying all offer paths to improve SE, yet each introduces trade-offs in complexity and overhead. As CR networks evolve toward 5G-Advanced and beyond, the continued refinement of FSK-based designs—especially when paired with AI-driven control—will ensure that this classic modulation remains competitive in the quest to maximize every hertz of the licensed spectrum.