Live data from Hacker News

Pixel Visual Core: Google’s first custom-designed co-processor

blog.google

91–100 of 102 posts

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#91
Is the HDR+ stuff running on the PVC already, or is it still running on the application processor until they enable it?

They say it uses Tensorflow, but is it a limited subset of Tensorflow designed only for image processing, like running a pretrained model on a TPU?

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#92
post #90
post #67

Maybe Google should start designing it's own CPUs/SoCs as well, since Qualcomm doesn't really seem capable of keeping up with what Apple is doing. Snapdragon 835 came out 6 months after the A10, and scored about 2/3 as well at single core, and less than 10% on Geekbench 4. A11 scored twice what the 835 did on single core, and over 50% more on multi core, and the 835 is going to be the best chip available in the U.S.…

The problem is that basically Qualcomm is strangling the entire cellphone space. So what if you built a completely new, much faster, much better-in-every-way SoC... if you still have to pay Qualcomm for the entire chipset because they refuse to sell you just the modem without you paying the full bill. Multiple designers could run circles around Qualcomm, but the marginal cost of doing so combined with the number of C…

> The problem is that basically Qualcomm is strangling the entire cellphone space. So what if you built a completely new, much faster, much better-in-every-way SoC... if you still have to pay Qualcomm for the entire chipset because they refuse to sell you just the modem without you paying the full bill.

If Qualcomm is that big of a drag on Android (or not-Apple, but it amounts to the same thing) smartphones, maybe the solution is for Google to buy them outright and rearrange their practices.

Sure, it'd be expensive, but not anything Google couldn't afford.

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#93
post #67

Maybe Google should start designing it's own CPUs/SoCs as well, since Qualcomm doesn't really seem capable of keeping up with what Apple is doing. Snapdragon 835 came out 6 months after the A10, and scored about 2/3 as well at single core, and less than 10% on Geekbench 4. A11 scored twice what the 835 did on single core, and over 50% more on multi core, and the 835 is going to be the best chip available in the U.S.…

Why not also blame Samsung for their Exynos or Huawei for their Kirin? Do you know what all 3 of these 3 SoC's have in common? They all use the same reference design developed by ARM. None of these SoC makers are going to spend the additional billions in SoC R&D deviating from the ARM design. If you really want to see a change then it must begin with ARM.

>Qualcomm doesn't really seem capable of keeping up with what Apple is doing. Snapdragon 835 came out 6 months after the A10, and scored about 2/3 as well at single core, and less than 10% on Geekbench 4. A11 scored twice what the 835 did on single core, and over 50% more on multi core, and the 835 is going to be the best chip available in the U.S. for Android until sometime next year.

Qualcomm and Apple are on different release cycles. The SD 845 will be in the same ballpark in multicore score and about 25% slower in single core than the A11.

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#94
Consider how inexpensive and high quality modern image sensors are. With this SoC, you can use so many amazing image processing algorithms. ML based object masking, deconvolution, focus stacking, exposure stacking, sensor fusion like FLIR, frame stitching multiple cameras, color compensating in camera flash. For serious heavy lifting you can send these bundles of images to the cloud for processing. All done without any more user input than tapping the take photo button.

It's going to seem like magic to the user.

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#95

Consider how inexpensive and high quality modern image sensors are. With this SoC, you can use so many amazing image processing algorithms. ML based object masking, deconvolution, focus stacking, exposure stacking, sensor fusion like FLIR, frame stitching multiple cameras, color compensating in camera flash. For serious heavy lifting you can send these bundles of images to the cloud for processing. All done without a…

A tag-a-long comment, this SoC + imaging system is going to make it much easier to write the software because you know all of the particulars of the optical system, sensor, position data, etc etc. It's not like implementing this stuff on the application layer which is nightmare mode difficult. Cameras are so fast, their controls are so fast, this is like some super human time dilation photography and photogrammetry.

I've held out for so long, it's finally time to replace my HTC One m7.

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#96
post #86
post #75

Earlier quoted context omitted.

I don’t think google has enough volume to make that worthwhile. Wouldn’t you need to sell tens of millions of chips for that to pay off?

If they only used it in their own devices, it would probably be too expensive. If they sold those devices to other manufacturers, they could probably get enough volume to at least avoid taking big losses. It's less about making money from selling the chips, and more about keeping the hardware behind the Android ecosystem competitive.

I wonder if the other OEMs would buy it. I could see Samsung wanting to make their own or at least not strengthen their dependence on Google.

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#97

Earlier quoted context omitted.

If you have burst mode and can get the raw frames, then shouldn't you be able to recreate the effect in post-processing? With the severe power constraints of a mobile phone a co-processor makes sense, but I'd think a general purpose desktop processor could easily handle this algorithm.

Can you burst 80 frames @ 60fps of 12 bit 12 megapixel RAW and save all that output, together with the output of gyroscopes and accelerometers timed to the start and end of exposure of every row of the image? If you can, you might be able to postprocess it... But as far as I know, no DSLR cameras even have gyros, nor sufficiently high burst rates, so that kind of photography is out of the question. It comes out to ov…

Good point about the gyros and accelerometers. I'd think you could do decent lossless compression given that the frames should very similar. I suppose though that once you are doing sophisticated compression you aren't that far off from implementing the post processing we are talking about.

I guess the long and short of it is these camera companies aren't software people--except isn't Sony a big camera company? You'd think they'd be able to bridge the gap.

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#98
post #88
post #55

Earlier quoted context omitted.

> This thread seems to be filled with amateur professionals thinking that their iPhone or Pixel will magically do something optically it can't (yet?). This is key. I own some decent camera gear and I'm amazed at what is coming out of all of the top end phones. Of course I can critique and find faults, but the pictures that non-photographers can take with their phones in a matter of seconds are down right amazing. I a…

> but the pictures that non-photographers can take with their phones in a matter of seconds are down right amazing. But that simply highlights that simply owning good gear doesn't mean you can shoot good pictures. You still need to learn lighting, composition, posing your subjects, etc etc, which was always the case!

Absolutely. Every photographer should be focused on those things as job one. My point above was that many wannabe photographers just focus on gear, and they are very quick to be critical of what phones are producing. My point was that phones can and are making great pictures without requiring a bag full of gear, and that it's highlighting that gear is secondary to the skills you listed.

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#99

Earlier quoted context omitted.

Halide is much more flexible. OpenCV is a fixed high level toolbox, combining things requires lots of intermediate buffers. Halide allows compiling arbitrary kernels in a fused way with extremely good performance. Sure you could implement OpenCV functions using Halide, but better to expose the lower level more flexible API. Source: I’ve rewritten parts of an OpenCV app in Halide for performance. Also your argument th…

That bet is the problem. Tensorflow on mobile already targets GPU and is going to be even more so with XLA. This debate is going to be hearsay unless we have an apples to apples comparison of the base tensorflow code running on the halide ir targeted to the tpu vs a vanilla ARM GPU... Even using opencv. I find it telling that they compared cpu vs tpu... Not GPU vs tpu.

I think you’re confusing two separate things. There’s two separate approaches to a computer vision task: classical and deep learning. Halide and OpenCV are for classical approaches, TensorFlow is for deep learning.

People don’t use both for the same problem, nobody is using “tensorflow code running on the halide ir”.

Both approaches use a lot of basic operations applied to matrices, so the custom ASIC supports either, but separately. HDR sounds like a better fit for a classical approach, so their HDR filter is probably written in Halide, with no machine learning or TensorFlow anywhere.

Re: Pixel Visual Core: Google’s first custom-designed co-processor

#100

Earlier quoted context omitted.

I'm really confused why you keep bringing up opencv. What does opencv have to do with the (presumably) TF ML model running on the IPU that you want to benchmark against the TF ML model running on a GPU?

Because halide is positioned as a high performance improvement over opencv. While I believe it is just an architectural decision to enable a non-portable ML model.

But Halide does have higher performance, is portable to many platforms as well as GPUs, CPUs (w/ parallel SIMD), and isn’t used for ML models...
Post reply on HN