Galaxy S23 Ultra's Secret Weapon Is This Samsung Camera Chip
Exclusive: Samsung's Isocell HP2 image sensor can shoot photos at 12.5, 50 or even 200 megapixels, thanks to more efficient light-gathering technology.
- Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
The Samsung S23 Ultra phone incorporates a new 200-megapixel image sensor for its main camera.
With a new 200-megapixel image sensor in its Galaxy S23 Ultra, Samsung is trying to give smartphone photographers the best of both worlds: a camera that takes extremely detailed photos and also works well with challenging lighting conditions that stymie many phones.
The sensor, called the Isocell HP2, was at the heart of Samsung's Unpacked 2023 launch event Wednesday, which touted photo and video performance as the S23 Ultra's most important new technology. Samsung detailed the sensor exclusively to CNET in January.
More on Samsung Unpacked
- Galaxy S23, S23 Plus Get Bigger Batteries and Faster Performance
- Samsung Galaxy Book 3 Ultra Takes On MacBook Pro
- Samsung Galaxy S23 Specs vs. S23 Plus and S23 Ultra: New Phones Compared
There aren't any HP2 samples to test yet, but Samsung's engineering improvements have a track record of improving image quality to address what's arguably the most competitive aspect of smartphones today.
More than a decade ago, Apple iPhones helped convince many people that their phone was the only camera they needed most of the time, but Google's image processing smarts helped its Pixel phones surpass Apple in some ways. Samsung, which manufactures its own image sensors, is now also competitive, in part by packing multiple cameras onto its phones.
The addition of the larger, 200-megapixel sensor in the Samsung Galaxy S23 Ultra "will provide substantial improvements in both low-light camera settings and resolution, offering advanced features that are still trading as significant points of differentiation in a sea of upgrade plainness amongst most premium competitor phones," said ABI Research analyst David McQueen.
Image sensor design tradeoffs
Image sensor designers face a tradeoff. Increasing sensor resolution means each pixel on the sensor is smaller, and for a given technology, smaller pixels aren't able to gather light as well. That means shots taken in low light are marred by noise speckles. They lose detail in shadowed parts of a scene. And they suffer blown-out highlights in bright areas like skies.
The HP2, though, brings new methods to counteract those problems and make the most of each photon of light, Samsung said. The HP2 is a bit smaller than its HP1 predecessor, but each pixel is more efficient at gathering light, an engineering direction that enables higher resolution photos and a smaller camera bump.
The South Korean electronics giant's sensor can gather light more effectively in the first place, boost high dynamic range (HDR) photos to cope better with scenes with dark and bright elements, the company said. And when shooting at the full 200-megapixel resolution, Samsung uses AI technology to help render the finest details.
It's no surprise Samsung is focusing on the technology. Camera improvements are a prime reason to upgrade phones, with better photos and videos more noticeable than marginally better processors, battery life and network technology.
"The full 200MP resolution especially shines when shooting at concerts or outdoors where there's lots of detail to be captured," said JoonSeo Yim, executive vice president of Samsung Electronics' sensor business. "It may not be the predominant setting for most consumers, but we definitely see the need for highly detailed images."
Apple, Samsung's top smartphone rival, is likewise investing heavily in its cameras. Comparatively large lens elements protrude from the back of iPhone 14 Pro models to show off camera performance, and Apple has upgraded its sensors for better high-resolution and low-light shooting.
Better pixel binning options
One of the headline techniques for improving smartphone photos is called pixel binning. With it, groups of physical pixels can be combined into larger virtual pixels that gather more light when it's dim, trading off resolution for lower noise and better color.
Samsung's Isocell HP2 image sensor takes photos at up to 200 megapixel resolution and adds new HDR tricks.
Samsung isn't alone in using pixel binning. You'll see it in the Apple iPhone 14 Pro, Google Pixel 7, Xiaomi 12T Pro and other phones, but the HP2 sensor is one of the most advanced. Apple and Google, for example, use 2x2 pixel binning that turn four physical pixels into one virtual pixel. Samsung's flagship Galaxy S22 phones have offered 3x3 pixel binning since 2019, offering 108 megapixel photos in good light and 12-megapixel photos when it's dim.
Samsung's HP2 can take 200-megapixel photos under good conditions. When it's dimmer, pixel binning groups pixels into 2x2 chunks for a 50-megapixel image. And when dimmer yet, Samsung's 4x4 "Tetra2pixel" chunks take a 12.5-megapixel photo.
The two levels of pixel binning were available on the 200-megapixel HP3, announced in 2022. However, the HP3 uses smaller pixels that, while minimizing camera bulk, aren't as good at capturing light in the first place. The HP1, announced in 2021, also had it. But the HP2 adds some other tricks the HP1 lacks.
Pixel binning ups and downs
Pixel binning has some other advantages. Cameras can crop in on the central portion of the image to zoom into more distant subjects. It's a key foundation to the effort to give smartphones zoom abilities like traditional camera lenses. Pixel binning also opens up new options for high resolution 4K and 8K video.
Pixel binning has downsides, though. It takes a lot of battery power to process all those pixels, and storing high-resolution photos gobbles up a lot of storage space. And high-resolution sensors, while nice in principle, don't achieve top image quality unless they're paired with high-quality lenses.
"The full 200MP mode does require more RAM and power," Yim said, which is why such high resolution sensors are only found on high-end smartphones.
One complication with the HP2 is figuring out color when shooting 200 megapixel photos. Digital cameras capture either red, green or blue light for each pixel, but the Tetra2pixel design means each 4x4 pixel group captures only one of those colors. To help fill in the color detail needed within those 16-pixel groups, Samsung uses an artificial intelligence algorithm, the company said.
Photography enthusiasts -- a population likely to be very interested in the sensor's high resolution -- can shoot raw images at 200 megapixels, Samsung said. Raw images are larger but offer higher image quality and better editing flexibility than conventional JPEG or HEIC photos.
Samsung HP2's image quality improvements
The sensor has other tricks to boost image quality, particularly with high dynamic range scenes with both bright and dark details. Here are a few:
- A technology called Dual Voltage Transfer Gate (D-VTG) gives each pixel a 33% better ability to gather light, which should improve image quality in dim scenes and cut back on washed-out white patches in bright skies.
- Samsung's Dual Slope Gain (DSG) feature improves HDR photos by digitizing each pixel's exposure data at two different scales to gather bright and dark data when shooting in 50-megapixel mode. The abundant pixels on the sensor mean some pixel quartets are tuned for bright light and others for dimmer light.
- A related feature called Smart-ISO Pro is a separate HDR technology that adapts to different scenes, employing different combinations of sensitivity settings appropriate for the different frames used to build the HDR photo.
Another new feature in the HP2 is an improved autofocus with a technology called Super QPD. It can spot either horizontal and vertical lines across 2x2 pixel groups, helping the camera lock onto details like horizons or tree trunks even when it's dim, Samsung said.
Each HP2 pixel is 0.6 microns, or 6 millionths of a meter, wide. That's a shade narrower than the 0.64 microns of the HP1, which means the HP2 is a slightly smaller image sensor at 11.3x8.4mm. For comparison, a human hair is something like 75 microns across. Combined into a 2x2 array for 50-megapixel photos, the pixel width increases to 1.2 microns, and in 4x4, to 2.4 microns.
"We expect that high-resolution image sensors will become a standard feature in future flagship smartphones," Yim said. "Because of that, we think it's important to continue our efforts, from advanced pixel processes below 0.5 microns to pixel performance and algorithms."
Larger sizes are better at gathering light. The Samsung pixel sizes are pretty similar to the iPhone 14 Pro's main camera sensor, which uses 2.44 micron pixels in 12-megapixel mode and 1.22 microns in 48-megapixel mode.
When it comes to video, the HP2 has many options. It can shoot 8K video at 30 frames per second by using the sensor in its 50-megapixel mode. It can shoot 4K video at 120fps, or, if Smart-ISO is engaged, at 60fps. For 1080p video, the sensor will shoot at 480fps without autofocus and 240fps with autofocus.
Correction, Jan. 18, 2:48 p.m. : This story misstated the pixel size on the Samsung Isocell HP1 image sensor. Each pixel is 0.64 microns across.