Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What an unprocessed photo looks like

maurycyz.com

Post reply on HN
Print view View original

21–30 of 418 posts

Re: What an unprocessed photo looks like

#21

Post by steveBK123 »

XCSme wrote: post #13

I am confused by the color filter step. Is the output produced by the sensor RGB or a single value per pixel?

In its most raw form, camera sensors only see illumination not color.

In front of the sensor is a bayer filter which results in each physical pixel seeing illumination filtered R G or B.

From there the software onboard the camera or in your RAW converter does interpolation to create RGB values at each pixel. For example if the local pixel is R filtered, it then interpolates its G & B values from nearby pixels of that filter.

https://en.wikipedia.org/wiki/Bayer_filter

There are alternatives such as what Fuji does with its X-trans sensor filter.

https://en.wikipedia.org/wiki/Fujifilm_X-Trans_sensor

Another alternative is Foveon (owned by Sigma now) which makes full color pixel sensors but they have not kept up with state of the art.

https://en.wikipedia.org/wiki/Foveon_X3_sensor

This is also why Leica B&W sensor cameras have higher apparently sharpness & ISO sensitivity than the related color sensor models because there is no filter in front or software interpolation happening.

Re: What an unprocessed photo looks like

#22

Post by wtallis »

XCSme wrote: post #13

I am confused by the color filter step. Is the output produced by the sensor RGB or a single value per pixel?

The sensor outputs a single value per pixel. A later processing step is needed to interpret that data given knowledge about the color filter (usually Bayer pattern) in front of the sensor.

Re: What an unprocessed photo looks like

#23

Post by delecti »

barishnamazov wrote: post #7

I love posts that peel back the abstraction layer of "images." It really highlights that modern photography is just signal processing with better marketing. A fun tangent on the "green cast" mentioned in the post: the reason the Bayer pattern is RGGB (50% green) isn't just about color balance, but spatial resolution. The human eye is most sensitive to green light, so that channel effectively carries the majority of t…

I have a related anecdote.

When I worked at Amazon on the Kindle Special Offers team (ads on your eink Kindle while it was sleeping), the first implementation of auto-generated ads was by someone who didn't know that properly converting RGB to grayscale was a smidge more complicated than just averaging the RGB channels. So for ~6 months in 2015ish, you may have seen a bunch of ads that looked pretty rough. I think I just needed to add a flag to the FFmpeg call to get it to convert RGB to luminance before mapping it to the 4-bit grayscale needed.

Re: What an unprocessed photo looks like

#24

Post by throw310822 »

delecti wrote: post #10
throw310822 wrote: post #2

Very interesting, pity the author chose such a poor example for the explanation (low, artificial and multicoloured light), making it really hard to understand what the "ground truth" and expected result should be.

I'm not sure I understand your complaint. The "expected result" is either of the last two images (depending on your preference), and one of the main points of the post is to challenge the notion of "ground truth" in the first place.

Not a complaint, but both the final images have poor contrast, lighting, saturation and colour balance, making them a disappointing target for an explanation of how these elements are produced from raw sensor data.

But anyway, I enjoyed the article.

Re: What an unprocessed photo looks like

#25

Post by XCSme »

steveBK123 wrote: post #21
XCSme wrote: post #13

I am confused by the color filter step. Is the output produced by the sensor RGB or a single value per pixel?

In its most raw form, camera sensors only see illumination not color. In front of the sensor is a bayer filter which results in each physical pixel seeing illumination filtered R G or B. From there the software onboard the camera or in your RAW converter does interpolation to create RGB values at each pixel. For example if the local pixel is R filtered, it then interpolates its G & B values from nearby pixels of that…

What about taking 3 photos while quickly changing the filter (e.g. filters are something like quantum dots that can be turned on/off)?

Re: What an unprocessed photo looks like

#27

Post by Forgeties79 »

For those who are curious, this is basically what we do when we color grade in video production but taken to its most extreme. Or rather, stripped down to the most fundamental level. Lots of ways to describe it.

Generally we shoot “flat” (there are so many caveats to this but I don’t feel like getting bogged down in all of it. If you plan on getting down and dirty with colors and really grading, you generally shoot flat). The image that we handover to DIT/editing can be borderline grayscale in its appearance. The colors are so muted, the dynamic range is so wide, that you basically have a highly muted image. The reason for this is you then have the freedom to “push” the color and look and almost any direction, versus if you have a very saturated, high contrast image, you are more “locked” into that look. This matters more and more when you are using a compressed codec and not something with an incredibly high bitrate or raw codecs, which is a whole other world and I am also doing a bit of a disservice to by oversimplifying.

Though this being HN it is incredibly likely I am telling few to no people anything new here lol

Re: What an unprocessed photo looks like

#28

Post by mmooss »

stavros wrote: post #6
userbinator wrote: post #5

I think everyone agrees that dynamic range compression and de-Bayering (for sensors which are colour-filtered) are necessary for digital photography, but at the other end of the spectrum is "use AI to recognise objects and hallucinate what they 'should' look like" --- and despite how everyone would probably say that isn't a real photo anymore, it seems manufacturers are pushing strongly in that direction, raising iss…

One thing I've learned while dabbling in photography is that there are no "fake" images, because there are no "real" images. Everything is an interpretation of the data that the camera has to do, making a thousand choices along the way, as this post beautifully demonstrates. A better discriminator might be global edits vs local edits, with local edits being things like retouching specific parts of the image to make d…

> What most people think when they say "fake", though, is a photo that has had filters applied, which makes zero sense. As the post shows, all photos have filters applied.

Filters themselves don't make it fake, just like words themselves don't make something a lie. How the filters and words are used, whether they bring us closer or further from some truth, is what makes the difference.

Photos implicitly convey, usually, 'this is what you would see if you were there'. Obviously filters can help with that, as in the OP, or hurt.

Re: What an unprocessed photo looks like

#29

Post by card_zero »

XCSme wrote: post #20
ranger207 wrote: post #18

Earlier quoted context omitted.

It's a single value per pixel, but each pixel has a different color filter in front of it, so it's effectively that each pixel is one of R, G, or B

So, for a 3x3 image, the input data would be 9 values like: R G B B R G G B R ?

In the example ("let's color each pixel ...") the layout is:

  R G
  G B
Then at a later stage the image is green because "There are twice as many green pixels in the filter matrix".
Post reply on HN