The new iPhones can change physics

Forums › Digital Imaging › The new iPhones can change physics

Batman

Mad Scientist
29761 posts

Uber Geek

Trusted

Lifetime subscriber

#240656 18-Sep-2018 22:05

I've been thinking about this for a while.

I saw a video of Apple showing how with their new iPhones you can alter the blurring of the background.

In that clip there is a slider where the numbers go between "f1.4" and "f16"

Firstly I hope that the slider is neutral at f2.4, because that is the aperture that you took the picture.

Now - all the other number must be fake.

I haven't done any calculations but the software effect you see cannot be f1.4. If i had a 50mm camera mounted onto a body with a full frame sensor, that is about the blurring you'd see at the comparative distance ratio of subject to background. With a 1/3" sensor, the f stop must be what 0.6?

The other side of the number slider. f16. So they are doing some fancy CSI sharpening of things that are out of focus or how does that slider work .... unless they already take pictures at every aperture on the slider which I doubt ...

Soon everyone will never understand photography.

Geektastic

17942 posts

Uber Geek

Trusted

Lifetime subscriber

#2092909 18-Sep-2018 22:31

It's hard to dissect computational photography.

Essentially it's a facsimile of the optical effects which is (probably) not as good as the real thing but cheaper and easier to carry.

It is possible that the phones are doing focus stacking, where in fact the image is composited from a number of images taken at a variety of focus points. I can do this on my Olympus EM-1 Mk2 where it can be useful in landscape photography if you want great depth of field but do not want the IQ reduced by diffraction errors at, say, f16 or f22. You shoot at f8 and use the automatic focus stacking setting. Also handy for macro work where DOF is very tight.

I would cheerfully lose the camera on my phone. I do not need one and if I do use it, it's only to take visual notes like images of written phone numbers or some such, or very occasional snapshots. For making photographs, I prefer to use the correct tools. It's kind of a deal - I don't use my camera to make phone calls...!

jarledb

Webhead
3254 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#2092911 18-Sep-2018 22:35

Yeah, its not real f stops. Its adjusting the blur added to the picture in a "magic" way.

Or to use Geektastics words: Computational photography.

Jarle Dahl Bergersen | Referral Links: Want $50 off when you join Octopus Energy? Use this referral code
Are you happy with what you get from Geekzone? Please consider supporting us by making a donation or subscribing.

Oblivian

7297 posts

Uber Geek

ID Verified

#2092925 19-Sep-2018 00:23

Correct about the lenses width throwing the focus plane out the window.

It seems the smarts/AI behind how the bokeh works on mobiles is still fairly hush-hush and or propriety between makers whether it uses software overlays or real smarts from the lens. But seem to make use of the BW lenses accross the board. Or video and lots of processor to do live software selecting/masking

Both cameras operate together to produce a single image with more data than would otherwise be possible to work with; the monochrome sensor is used to pick up on more detail. In portrait mode, the two lenses create a 3D image of the subject’s face, and use this to work out which parts of the image should be in focus.

That’s how the iPhone 7 Plus system works, too, but the camera setup is very different. Apple paired one 28mm-equivalent f/1.8 lens with a 12-megapixel sensor for the primary camera, while the secondary camera has a 56mm-equivalent f/2.8 lens without stabilization over a slightly smaller 12-megapixel sensor.

Coil

6614 posts

Uber Geek
Inactive user

#2092930 19-Sep-2018 07:12

Physics in the term that the retail significantly outweighs the cost?

amiga500

1484 posts

Uber Geek
Inactive user

#2102158 5-Oct-2018 13:33

When Nikon brought out the Nikon D3200 with the 24 mp sensor they encouraged people to crop photos a little when using the kit 18-55 mm lens. Nikon said because of the high mp some cropping could be done without loss of quality - and they seemed to be right. So, really it's what the eye can discern that matters.

SpartanVXL

1306 posts

Uber Geek

#2102174 5-Oct-2018 13:46

With Live photos being enabled on newer iphones it's quite possible that multiple frames are used to composite the final image. It's not entirely a gimmick, some shots end up quite a bit better. Some do look hilariously bad when the focus is wrong however.

gehenna

8497 posts

Uber Geek

Moderator

Trusted

Lifetime subscriber

#2102180 5-Oct-2018 13:52

This is an excellent primer on how the XS camera works: https://blog.halide.cam/iphone-xs-why-its-a-whole-new-camera-ddf9780d714c

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

Handsomedan

7284 posts

Uber Geek

ID Verified

Trusted

Subscriber

#2103177 8-Oct-2018 09:53

gehenna:

This is an excellent primer on how the XS camera works: https://blog.halide.cam/iphone-xs-why-its-a-whole-new-camera-ddf9780d714c

I read this on Reddit.

There's a lot of good (and bad) conversation that has ensued since the original post and the creator of Halide has done their best to explain in detail what/why/how etc...

Handsome Dan Has Spoken.
Handsome Dan needs to stop adding three dots to every sentence...

Handsome Dan does not currently have a side hustle as the mascot for Yale

*Gladly accepting donations...

gehenna

8497 posts

Uber Geek

Moderator

Trusted

Lifetime subscriber

#2103179 8-Oct-2018 09:56

Some people can't get past their biases.

Jaxson

8042 posts

Uber Geek

Trusted

#2103197 8-Oct-2018 10:31

Brother has one of the iPhone Xs, and frankly it's pretty darn good. Given you can do a lot of this stuff in Photoshop/lightroom afterwards, it doesn't seem that crazy that it can't be incorporated into the original image capture/generation process.

DSLR's etc are looking quite old fashioned now really, despite that image quality...

danza

112 posts

Master Geek

#2148901 21-Dec-2018 15:34

In simple term is that the phone uses dual camera or dual/focus pixels to capture two separate images that's one at a slightly different angle from another to create a depth map and apply blur accordingly.

Someone already posted about the dual camera, so here's dual pixel/on-sensor phase detection pixel, basically what all the new sensors in google pixels or iphones use to focus.

https://www.canon.co.uk/for_home/product_finder/cameras/digital_slr/dual_pixel_cmos_af/

Here's how google use a single camera/dual pixel to achieve the out of focus blur.

https://ai.googleblog.com/2017/10/portrait-mode-on-pixel-2-and-pixel-2-xl.html

It's no magic, just science and a lot of hard work.

gehenna

8497 posts

Uber Geek

Moderator

Trusted

Lifetime subscriber

#2148936 21-Dec-2018 15:47

danza:

It's no magic, just science and a lot of hard work.

As referenced above here: https://blog.halide.cam/iphone-xs-why-its-a-whole-new-camera-ddf9780d714c

danza

112 posts

Master Geek

#2149062 21-Dec-2018 20:47

gehenna:
danza:

It's no magic, just science and a lot of hard work.

As referenced above here: https://blog.halide.cam/iphone-xs-why-its-a-whole-new-camera-ddf9780d714c

Yeah that app looks decent and bypass the issue of the Xs using a faster shutter speed (to compensate for most people's unsteady hands). But they are also just comparing JPEG (that is the smart hdr) vs raw, which is already a known thing.

Those are good steps towards better images but I thought we were talking about the depth of field effect :v