Recent news suggests Apple is laying the groundwork for a mobile camera capable of freezing an entire scene in an instant. 

This is known as a global shutter, as confirmed by a new patent. This technology promises clearer motion, less streaking, and more professional-looking videos. Here's a quick explanation that anyone can easily understand.

Imagine the referee shouting "Cut." Everyone stops at once. The camera then records everything it sees. Because everyone stops at the same time, straight lines remain straight. LED walls and stage lights are more effective. "Flashlights" make it easier to capture the moment of play. Here are some of the principles:

1. A rolling shutter reads the image line by line. Rapid motion causes straight objects to bend and create jitter.

2. A global shutter captures every pixel at the same moment and then reads the data.

3. Apple's patent shows a tiny "buffer" inside each pixel, allowing the phone to freeze an entire frame at once. This is a brilliant idea. Based on this application principle, you can refer to the following solutions:

Next, we'll explain Apple's patent in layman's terms.

First, imagine each pixel as a three-layer stack. The first layer is where the light comes in, the second layer is a small buffer, and the third layer is a tiny meter that converts the light into a numerical value.

Here's how it works: During exposure, the first layer is filled with light. At the frozen moment, each pixel simultaneously shifts light from the first layer to the second layer, thus freezing the entire image. The phone then shifts this stored light from the second layer to the third layer, line by line, to create the photo. The secret lies in the middle layer, which provides each pixel with a dedicated "resting point" to store the frozen image.

So, what's different about Apple's approach in this scenario?

As shown in the image, Apple's approach is stacking the sensors upwards, rather than horizontally. Many designs currently on the market place the "buffer" next to the spotlight, which takes up space and light. Apple stacks the channels vertically, saving even more space for light collection.

A single component performs two tasks—the middle layer stores the image and helps move it. This makes the pixels compact and easy for the phone to operate. It also reduces stray light in still images. The patent describes a shield and groove structure that protects the "buffer" when the shutter is "closed." This results in sharper still images and clearer images. As shown below, the drawings include backside illumination and split pixel technology for autofocus. These are core components of high-density smartphone sensors, not bulky laboratory parts.

Apple's own description of the patent

As Apple explains, an image sensor may include multiple pixels, each of which may include a photodiode having a charge accumulation region ("PD"), a floating diffusion region ("FD"), and a charge transfer region vertically located between the PD and the FD. The vertical charge transfer region may include a first charge modulation region ("P1"), a second charge modulation region ("P2"), and a third charge modulation region ("P3"). The image sensor may operate in global shutter mode, where P2 may serve as an intra-pixel charge storage region for temporarily storing charge while it is transferred from the PD to the FD via P1, P2, and P3.

This patent generally relates to an image sensor, and more specifically to an image sensor pixel technology with vertically integrated multi-phase charge transfer technology for image capture in global shutter mode.

Apple notes that image capture devices are widely used in various electronic devices, such as mobile devices (smartphones, tablets, laptops, etc.), robotic devices, or security monitoring equipment. An image capture device may include an image sensor having multiple light-collecting pixels. Each pixel may include a photodiode. The image capture device may capture light from the environment and transfer it to the image sensor. When exposed to light, a pixel's photodiode can accumulate charge. During readout, the charge in the photodiode can be read out from the photodiode using one or more transistors to generate an analog image signal. The analog image signal can be converted to a digital signal and further processed to generate an image.

As Apple explains, in some instances, an image sensor can include multiple light-collecting pixels, for example, organized into a pixel array having one or more rows and one or more columns of pixels. In some instances, the image sensor can be a CMOS (complementary metal oxide semiconductor) image sensor, a CCD (charge coupled device) image sensor, or the like. In some instances, the image sensor can be part of an image capture device (e.g., a camera), which can also be part of an electronic device (e.g., a smartphone, tablet, laptop), a robotic device, or a security monitoring device. In some instances, each pixel of the image sensor can include at least one photodiode, which includes a charge accumulation region (hereinafter referred to as "PD"), a floating diffusion region (hereinafter referred to as "FD"), and a charge transfer region vertically located between the PD and the FD. "When each pixel is exposed to light, the PD can accumulate charge, or photocarriers. During readout, at least some of the charge can be transferred from the PD to the FD, thereby generating an analog image signal (e.g., an analog voltage) on the FD, which can be further accessed via a pixel output line external to the pixel. In some embodiments, the analog image signal accessed via the pixel output line can be further processed, for example, by analog-to-digital conversion using an analog-to-digital converter, and then digitally processed by an image signal processor (ISP) to generate one or more images," Apple emphasizes.

Apple further notes that, in general, a given image acquisition device can operate in rolling shutter mode or global shutter mode. In rolling shutter mode, different rows of the image sensor pixel array of the image acquisition device are exposed at different times as a readout "wave" sweeps across the image sensor. For example, pixels of the pixel array can be exposed, and the image signals of the pixels can be read sequentially, for example, row by row from the top to the bottom of the pixel array. For example, pixels in the same row can be read out simultaneously, while pixels in the same column but not in the same row can be read out sequentially, one by one. Therefore, in rolling shutter mode, an image sensor can record an image sequentially, row by row, rather than capturing the entire image all at once.

In contrast, in global shutter mode, the exposure time is the same for all pixels, meaning that each pixel in the image sensor can begin and end exposure simultaneously. Therefore, the entire image can be recorded at once. Because different "lines" of the image are recorded at different times, rolling shutter can cause variations in the color and/or tonality of the captured image. In certain applications, such as high-speed photography or video recording, this can cause significant noise and significantly impact the quality of the captured image.

Thus, in some instances, a global shutter may be preferred. However, in some instances, even if the image sensor's pixels end exposure simultaneously, their image signals may still be read sequentially, for example, row by row, as in a rolling shutter mode. Therefore, the image sensor may require a "memory" to temporarily store the pixel's (a) charge (e.g., in the charge domain) and/or (b) analog or digital image signal (e.g., in the voltage domain) at the end of exposure until the individual pixel is read.

To address this issue, in some instances, each pixel of the image sensor disclosed herein may include an in-pixel charge storage region. At the end of exposure, charge can be transferred from the pixel detector (PD) to an intra-pixel charge storage region. The charge can be temporarily stored there until the pixel is read. During readout, the charge can be transferred from the intra-pixel charge storage region to a fluorescence detector (FD), from which analog image information can be further accessed via the pixel output line.

"One skilled in the art will appreciate that the image sensor disclosed herein can provide several advantages," Apple states.

First, it can provide an intra-pixel charge storage region for each pixel to temporarily store the pixel's charge, enabling the image sensor to operate in global shutter mode. Furthermore, a "memory" is integrated as part of the pixel to store charge internally, eliminating or at least reducing other additional storage components (e.g., memory chips on the image sensor). This can reduce the number of components in the image sensor, reduce the image sensor's footprint, and/or increase the sensor's pixel density.

Apple emphasizes that the methods described herein can, in various embodiments, be implemented in software, hardware, or a combination thereof. Furthermore, the order of the method blocks can be changed, and various elements can be added, reordered, combined, omitted, modified, etc. Those skilled in the art, having the benefit of this disclosure, may make various modifications and alterations. The various examples described herein are intended to be illustrative only and not limiting. Numerous variations, modifications, additions, and improvements are possible. Thus, multiple instances of the components described herein may be provided, rather than a single instance. Boundaries between various components, operations, and data stores are somewhat arbitrary, and specific operations will be described in the context of specific illustrative configurations. Other allocations of functionality are contemplated and may fall within the scope of the following claims.

Finally, structures and functionality presented as discrete components in the example configurations may be implemented as combined structures or components. These and other variations, modifications, additions, and improvements may fall within the scope of the embodiments defined in the following claims.

Source: Semiconductor Industry Observation

View more at EASELINK