Performance enhancement of optical camera communication system using optical camera communication coding and region ‐ of ‐ interest detection

Optical camera communication (OCC) is considered an important transmission scheme in implementing indoor navigation, beacons, and other Internet of Things applications. One of the disadvantages of OCC, however, is that transmission is successful only if the line ‐ of ‐ sight requirement is satisfied between transmitting LED lamps and receiving image sensors. Any obstacle between them or movement of the receiver direction causes a reduction in the light signal and leads to a burst error, and its compensation is not easy. A low ‐ density parity check (LDPC) code is employed in the OCC to overcome burst error. To improve the performance further, a region of interest area detection is proposed that utilizes only the centre part of an image frame for LDPC decoding rather than using the whole area measurement. The LDPC codes are tested using an appropriate OCC packet structure, and its performance is compared with that of polar codes. The experimental results indicate that LDPC coding is more suitable in OCC systems. It is found that the proposed LDPC coding scheme not only recovers the data from burst errors but also enhances the transmission distance. In particular, soft ‐ decision LDPC decoding shows more resistance to burst errors and bit loss.


| INTRODUCTION
With its property of large bandwidth, insensitivity to electromagnetic interference, and easy accessibility using light-emitting diode (LED) lamps, visible light communication (VLC) has emerged as a potential technology in implementing the Internet of Things (IoT) [1][2][3][4]. Optical camera communication (OCC) is a field of VLC that uses an image sensor as a receiver [5][6][7][8][9][10][11]]. An OCC system is expected to be used in indoor localizations [9] that GPS signals cannot reach and in transmissions between vehicles [10,11]. Furthermore, as more cameras are installed in equipment such as smartphones, cars, robots, drones, and home appliances, it is expected that OCC technology will play an important role in the provision of IoT infrastructure.
Implementation issues of OCC include flickering, data rate, and transmission distance. Flickering can be avoided by using a modulation frequency higher than 100 Hz [12] and line coding such as Manchester or 8B/10B coding. The data rate and transmission distance can be enhanced by using higher shutter speeds, better optical lenses, and larger-sized light sources [5]. However, unlike photodiode-based VLC systems, the bit loss problem of OCC is unavoidable. This happens because of the working process of a complementary metal oxide semiconductor (CMOS) image sensor when it is used in video mode. One of them is a variable interframe gap (IFG) [7], which is the time gap between two consecutive frames during which signals are not detected. CMOS image sensors have seen a remarkable increase in use compared with charge-coupled device (CCD) sensors due to their low price and easy integration into smartphones and other mobile devices. The major difference between these two technologies is the method each sensor uses to obtain information from its pixels. While all pixels of a CCD sensor are exposed simultaneously, CMOS sensors expose and read each pixel line in sequence. This mechanism is known as a 'rolling shutter'. In this process, the next frame must wait for a while to guarantee that the readout process of the current frame is completed before starting the next frame. This causes an IFG, which is not a problem in recording images but causes bit loss when the image sensor is used in communications. Another source of bit loss in OCC is a reduced light signal at the edge of an image sensor. Due to the CMOS sensor's shutter mechanism, the light intensity at the edge of the frame is weakened, which becomes more serious as distance is increased, as indicated in [5]. Another origin of signal loss is impairment in line-of-sight transmission due to obstacles or inadvertent movement of the mobile device, which leads to a stream of error bits called a burst error. To compensate for these problems, appropriate error correction coding is required.
In recent years, low-density parity check codes (LDPCs) have attracted growing interest [13,14]. LDPC codes can achieve performance very close to the constrained input Shannon limit, differing by only 0.0045 dB as demonstrated in [15]. LDPC codes have already been adopted in several wireless standards including by IEEE 802.11, digital video broadcast, and the Advanced Television System Committee [16,17]. Recently, LDPC codes were selected for the enhanced mobile broadband data channels for 5G New Radio [18]. Compared with turbo codes, which are the primary coding scheme in third-and fourth-generation (3G and 4G) mobile radio systems, LDPC codes achieve better performance and faster decoding [19,20]. Since the computation of LDPC codes is processed as separate parallel units, it can achieve hardware efficiency. Compared with Hamming codes, which are proposed for improving the performance of the OCC system in [7], LDPC codes can achieve much higher performance. Since a Hamming code corrects only one error per codeword, it cannot overcome a large IFG or burst errors. In [21], an efficient coding scheme using a polar code is proposed for a VLC system. It has the advantage that the calculation time is relatively small for a long code length, and the performance is slightly higher than that of LDPC codes in an additive white Gaussian noise environment. However, the performance of polar codes with short to moderate lengths or with high code rate is inferior to LDPC codes, as shown in [22,23]. The performance of LDPC and polar codes is compared in the presence of burst errors, which happen easily in an OCC system. Since the OCC packet length is limited due to the IFG, the advantage of polar codes is reduced in an OCC system, as demonstrated in Section 3. Moreover, LDPC codes are proposed to improve the performance of an OCC system against burst errors and bit loss. The performance of soft-decision LDPC decoding is also compared with that of hard-decision decoding.
In an attempt to further enhance performance, the measurement area is changed from the entire area of the image sensor to the region-of-interest (ROI) area to achieve a more appropriate threshold level for data decision. By applying both LDPC coding and ROI-area detection, the performance of OCC is measured to be considerably enhanced compared with the case without them. LDPC codes and the data packet structure along with the ROI-area detection are introduced in Section 2. The experiment results are explained in Section 3. Finally, the conclusions of this study are presented in Section 4.

| Why low-density parity check coding in optical camera communication?
LDPC codes were first proposed by Gallager in his 1960 doctoral dissertation [24]. After his work, other studies of LDPC codes were presented in [25][26][27]. Like other block codes, LDPC codes are represented by a parity-check matrix H, which has a low density of 1's as shown in Figure 1(a). A Tanner graph represents the parity-check matrix in the form of a bipartite graph [25] where variable nodes v are connected to check nodescas in Figure 1 In an OCC system, a burst error occurs when the camera cannot see the LED lamp because of an obstacle between them or a sudden movement of the device. Another type of error comes from the bit loss between two consecutive frames or the reduced light intensity at long-distance transmission. Transmitting data multiple times can avoid the lost bits in the IFG, but it cannot overcome the burst errors due to its random position and large size. Therefore, LDPC codes are an appropriate candidate in this environment. The data recovery function of LDPC codes against a burst error is explained in Figure 1. Each variable node is connected to multiple check nodes at different positions, as shown in Figure 1(b). Therefore, multiple check nodes will receive wrong information only when a variable node is wrong, as indicated by the red lines. However, as each check node is connected to multiple variable nodes at the same time, the aggregated information is still reliable assuming that most of its connected variable nodes are valid. Then, the information from the check nodes is sent to the variable nodes. Simultaneously, the information of other trusted nodes supports the error nodes to retrieve the information. The decoding process is repeated until the preset number of iterations is reached or all the data is completely recovered. Therefore, system performance depends on the number of iterations and coding efficiency. A large-size LDPC block can achieve high performance but at the cost of complexity. There are two types of iterative decoding algorithms: hard decision and soft decision. Hard decision is simple and easier to implement, while soft decision achieves better performance. LDPC codes with hard decision are sufficient against white noise, while soft decision is required to overcome the harsh burst error environment. Therefore, the appropriate decoder type should be selected considering both the environment and complexity. Both hard and soft decisions are applied to LDPC decoding of OCC, and their performance is evaluated.

| Structure of a data packet
The structure of a proposed data packet is illustrated in Figure 2, where data is LDPC-coded for error correction and Manchester-coded for flicker-free illumination. This data is transmitted twice in the form of a subpacket, where start frame (SF) bits are added at the beginning of each subpacket. The number of subpackets in each payload depends on distance, the modulation scheme, and reliability. From the principle of the rolling-shutter mechanism of CMOS image sensors, there is a variable time gap called IFG between two consecutive frames where bit loss happens. In order to prevent this bit loss, data should be transmitted multiple times so that at least one of the subpackets can avoid the IFG. The number of data repetitions is determined by the size of the IFG. The number of bits in a subpacket must be larger than the number of bits lost in the IFG. Simultaneously, the total number of bits in a packet must be less than the maximum number of bits in a frame to prevent subpackets of consecutive frames from meeting with IFGs. The maximum number of received bits per frame N frame can be calculated as where R b is the data rate [bps] and R frame is the frame rate [fps] of a camera. Based on the structure of the data packet, the pure information rate R info [bps] can be calculated as where N pk is the total number of bits in a packet, N SF is the number of bits in an SF, n is the number of data repetitions, and η LDPC and η Man ¼ 1=2 are the efficiency of LDPC and Manchester coding, respectively. Although the pure information rate is reduced with LDPC coding, the system becomes robust against background noise and burst errors.

| Region-of-interest-based decoding process
In an OCC system with a rolling-shutter mechanism, a bit is sampled many times, and each sampled value corresponds to the light intensity of a pixel line. The number of pixel lines per bit (L bit ) is decided by the frame rate (R frame ), vertical resolution (L frame ), and bit rate (R b ) as follows: In practice, however, the number of pixel lines representing a bit (L bit ) changes according to a threshold determined by the received signal at the camera. Because the received signal of the camera varies with the transmission distance or other environments, the threshold value for determining a 1 or a zero is also changing. There are several schemes for setting the threshold [28,29]. A polynomial regression scheme is used [5][6][7] in which average values for each line of pixels are calculated using a polynomial regression and used as the threshold to decide level 'high' or 'low'. When the same 'high' or 'low' level is measured L bit times, a '1' or '0' is decided, respectively. If the number of continuous 'high' or 'low' lines (L cont ) is much larger than L bit , the repeated bit numbers (B cont ) are calculated as follows: Actually, the value L cont can vary according to the threshold, and therefore, so does the system performance. In an OCC system, the received signal intensity is not constant over the whole sensor area; the centre area usually has a higher intensity than the other area in the frame. This comes from the 'blooming' effect due to the optical system of the CMOS sensor [28]. Unlike existing schemes that use the whole parts of a frame in decoding [5][6][7][8], we propose employing only an ROI part that uses only the centre part of the image in a frame. As data is transmitted twice, more than half a frame is enough to recover all the data. Therefore, the size of an ROI can be defined as L ROI ≥ 0:5L f rame . In other words, the size of the ROI is more than a subpacket. It is shown in Figure 3 that at least one SF and one payload must be included in an ROI. Therefore, all transmitted data can be recovered without using the whole data in a frame. It is seen that at least two subpackets are required to overcome the IFG problem. As the transmission distance is increased, the number of repeated subpackets should be increased to compensate for the reduced image size at the cost of the data rate.
In practice, however, the position of the LED pattern is not always located at the centre of a frame because of camera movement. Four different positions of LED pattern are shown in Figure 4, where sampling is performed from left to right. While all signals are received appropriately in Figure 4(a), a part of the signal is lost in Figure 4(b). In Figure 4(c), the signal is significantly reduced because of the shift of the image in the upper direction. As the LED pattern is moved towards the corner of the frame as in Figure 4(d), part of the signal is lost, and the amplitude of the remaining signal is reduced. This is considered the worst case in an OCC system. If the whole frame is used in the decoding in this case, burst errors will occur at the edge. If the position of the LED pattern is constantly changing, the ROI is chosen adaptively so that it can include a whole subpacket and achieve maximum possible amplitude. The algorithm for the selection of the ROI is described below and is demonstrated by experiments in this study.

Algorithm on the selection of the ROI
(1) Initialization: Find the number of pixel lines in a frame: L frame Find the pixel line with maximum amplitude: i max Find the number of pixel lines included in the ROI: L ROI Read all pixel data in a frame: data frame [1 × L frame ] (2) Selection of ROI data: data ROI If only the ROI is considered in decoding, this helps in finding more appropriate threshold levels, which leads to better performance. Figure 5 shows two types of threshold levels using a polynomial regression, one from the whole frame and the other from the ROI. Because the light intensity received at the image sensor varies across the frame, the threshold level using the whole data is significantly curved, as indicated by the black line, which can induce many error bits. On the other hand, the threshold level generated from the ROI becomes nearly linear and provides a relatively precise criterion in the narrow region.
It is assumed that L cont ¼ M ⋅ L bit � Δn, where M is the number of consecutive '0' or '1' bits and Δn is a fluctuation. The value ðL cont =L bit Þ in (4) can be rewritten as follows: If the value Δn/L bit is larger than 1, an error occurs. This happens easily when the number of pixel lines representing a bit, L bit , is small. In other words, the occurrence of errors becomes more frequent with a high data rate and low image resolution. This is attributed to the random starting position of a data bit in a line of pixels. In this harsh environment, ROIbased detection brings better performance than whole-area based detection does. The performance of the proposed ROI-based detection is demonstrated by the experiments in the next section.

| Hard-decision and soft-decision in LDPC decoding
In this system, on-off keying modulation is used for LED transmission, where only two levels of signal, that is, 'ON' and 'OFF', are used in the transmission. At the receiver side, however, distorted signals are measured, as shown in Figure 6. Soft-decision LDPC decoding requires signal values expressed in [−0.5, 0.5], reflecting the received light intensity. Since the ON/OFF period of the light source is represented by multiple pixel lines due to the rolling-shutter mechanism [30], the signal amplitude in each bit for the soft decision is determined by a group of pixel lines, as shown in Figure 6.
In order to calculate the value of each bit, multiple points are measured and averaged in each bit period. Once the threshold level is determined, it is subtracted from the measured values so that the average is set to zero. The number of consecutive pixel lines larger or smaller than the threshold per each bit, L cont , is counted. The average value S i representing the amplitude of a bit is calculated as where x j is the received light intensity of the j th pixel line, L cont;i corresponds to the number of pixel lines for the i th bit, and corresponding sampled signals for soft decision are illustrated in Figure 7(a) and Figure 7(b), respectively. These sampled signals are normalized to have amplitude in the range [−0.5, 0.5] to be used in LDPC decoding. Then, it is Manchesterdecoded by taking the front half value of the signal as shown in Figure 7(c), which will be fed to a soft-decision LDPC decoder to estimate the information data. For hard decision, the received light intensity is converted to binary numbers '0' or '1' by comparison of its level with the average value and Manchester-decoded as indicated in Figure 7(d). The front half value of the signal in this case is also used in Manchester decoding. Normally, LDPC decoding with hard decision is easier in implementation, while its performance is inferior to that of LDPC decoding with soft decision.

| EXPERIMENTAL RESULTS
To test the performance of the proposed scheme, the OCC system and LDPC coding/decoding algorithms are implemented. At the transmitter, an FPGA board is used in encoding the data, which is then sent to a LED driver where the encoded data is converted to the current that controls the light intensity of the LEDs. At the receiver, a CMOS image sensor of a digital camera (FLEA-3) is used to convert the light intensity into an electrical signal that is analyzed by a computer as illustrated in Figure 8. For encoding, a (648,324) LDPC parity-check matrix is used, which is obtained from an irregular quasi-cyclic LDPC code with code matrix size of 12 � 24 and cyclic permutation matrix of size Z � Z with Z = 27 [31]. Although effective LDPC codes are given in [32], the code size is very long compared with the LDPC code proposed in this study. As the data rate of the OCC system is low, that is, several kbps, a long block code will cause significant packet delay. A quasi-cyclic LDPC is used in this study because of its properties such as fast encoding, resizing of the parity check matrix without changing the LDPC code matrix, and easy changing of its code rate [31,33]. The sum-product algorithm (which computes the likelihood ratio for each bit) is used with a maximum of 30 iterations for decoding. For comparison, polar codes with a size of (512,256) and (1024,512) are used while a successive cancelation list decoder is employed. Each packet is composed of 136 bits that include two subpackets (60 bits per subpacket) and two SFs (8 bits per SF). The data rate is 10.8 kbps, while the information rate is 1.19 kbps. The detailed parameters used in the experiment are listed in Table 1. Bit error rate (BER) measurements of OCC with and without LDPC coding are shown in Figure 9. Signals are detected only in the ROI, and hard decision is used in the decoding of this experiment. It is shown that the performance of the scheme with LDPC is much better even though hard decision is used; its transmission distance is increased by 1.5 m or 1.7 dB at a BER of 10 −4 .
The pure information rate of LDPC coding is reduced by half due to its coding process. For a fair comparison, data rates are adjusted in the following experiments so that each scheme has the same information rate. Figure 10 shows the BER comparison among different detection schemes, that is, detection areas and detection types. It is shown in the figure that if LDPC-coded data is detected in the whole image frame, its performance enhancement is not improved much compared with the case without LDPC measured in the ROI. However, a significant improvement is found when both LDPC coding and ROI are used; an increase of about 0.5 m (0.5 dB) is seen in Figure 10. LDPC with soft decision increases this further. The difference in transmission distance between soft and hard The performance improvement with LDPC coding is found in the presence of a burst error that occurs from obstacles between the LED lamps and the detector, or deviation from lineof-sight transmission due to quick movement of a device. A burst error is a group of incorrect bits composed of contiguous zeros or 1, as illustrated in Figure 11. Although it may include some correct bits, that is, n c bits in the figure, they are regarded as a part of the burst error. The size of a burst error is indicated as a ratio of the total burst error bits in each frame.
Experiments in this study show that performance degradation is very sensitive to the burst error ratio, as shown in Figure 12, showing that the OCC scheme without coding cannot overcome the burst error. Although polar codes have good performance against low signal-to-noise ratios, their performance in the presence of burst errors is not as good as that of LDPC codes. It is measured that polar codes can overcome only a small-sized burst error, that is, less than 20% at a BER of 10 −3 . Although interleaving is helpful for enhancing the performance of polar codes against burst errors, its performance is still lower than that of the LDPC codes, as indicated in Figure 12. However, LDPC codes can overcome a large burst error size. An error-free performance is achieved with hard-decision LDPC for a burst error less than 20% of the image frame. The limit of the burst error size is increased to 25% when soft-decision LDPC is used. It is shown that the allowable burst-error ratio at a BER of 10 −3 is 25% and 35% with hard decision and soft decision, respectively. Therefore, LDPC codes are suggested for the OCC system, especially when burst errors are of concern.
In the following experiment, the performance of LDPC coding is evaluated when the image size of a rolling-shutter pattern is reduced because of long transmission distances [5,7]. In proportion to the distance, the image size and number of received bits are reduced. Figure 13(a) Figure 13(b) show the image at 3.6 m and 6.8 m, respectively. In the worst case, the number of received bits is not enough for data recovery. In this case, arbitrary or punctured bits are inserted in the position of lost bits as illustrated in Figure 13(c) [7]. For the scheme without error-correction coding, these lost bits are considered errors. For the scheme with LDPC coding, however, these lost bits can be recovered, and its performance depends on the length of the lost bits. Figure 14 shows the experimental results in the presence of this bit loss from a reduced image size. The measurement distance is 7 m, where error-free performance is achieved without a burst error, as shown in Figure 14. At a BER of 10 −3 , soft decision requires only 42% of the frame for detection, while hard decision and non-LDPC require as much as 48% and 57%, respectively, showing the enhancement of transmission distance with LDPC coding.

| CONCLUSIONS
We propose a scheme of LDPC channel coding and ROI-area measurement to improve the performance of OCC. It is well known that LDPC coding works effectively against an interference environment at fast decoding speeds. We introduce this scheme for OCC to compensate for a burst error that happens easily in OCC. The performance of LDPC coding against a burst error is measured and compared with that of polar codes. It is found from the experiment that LDPC-coded data can be recovered at a BER of 10 −3 even when the ratio of a burst error reaches 25% and 35% of each frame with hard and soft decision, respectively. As the distance becomes longer, the size of the rolling-shutter pattern at the image sensor is decreased, and performance is degraded. LDPC coding works effectively in this case also-detection is successful with soft-decision LDPC even when only 45% of the frame receives the OCC signal. It is found that the proposed ROI-area detection further enhances the performance of OCC. This scheme generates a more precise threshold level for bit decision than that of previous methods because only the centre lines of an image frame, where signals have uniform amplitudes, is used in the detection process. This ROI scheme extends the transmission distance by 0.5 m (0.5 dB) at a BER of 10 −3 .
It is demonstrated that the proposed LDPC coding scheme can secure the performance of OCC against burst errors. The length of the LDPC code can be selected that considers the channel environment and decoding time. The proposed scheme is considered quite useful in deploying an OCC system in a harsh channel environment. F I G U R E 1 3 Rolling-shutter image pattern at different distances with bit loss (a) At 3.6 m (b) At 6.8 m (c) Occurrence of lost bits against combination of two subpackets F I G U R E 1 4 Performance of LDPC coding against bit loss from reduced image size. LDPC, low-density parity check