Computers can do incredible things, but sometimes the things they work on still need a human touch.
Trends
For a little while, an app that clarified blurry images scattered across TikTok, Tumblr, Instagram, and any other site with picture-hosting ability. It really was incredible – eyes, noses, and mouths obscured by low-grade camera blur suddenly appeared in high depth once again. It sometimes saw eyes where there weren’t any, but that was pretty rare. The human face has a pretty standard formula, after all!
And then, when everyone had put their pictures of their grandparents through, a bunch of people did it to emojis and anime/cartoon stills, and the true magic of the app was unveiled. Just like Google Deep Dream, the app wasn’t actually seeing and understanding a face, it was just making guesses as best as it could. If a character had facial scars or markings, it had no idea what it was looking at, and would sometimes add an eye or mouth.
It might not have understood those on a regular human face, either, but the era of facial tattoos and the era of bad cameras didn’t overlap much, so it wasn’t as much of a concern.
Low-quality screengrabs of animations, however, are everywhere. Check out surprisingly well-made photo realistic Doom Guy on this guy’s Twitter.
Meanwhile, Sasuke from Naruto looks like a mess! It has no idea where the eyes are actually supposed to end. His original face is much more stylized than even Doom Guy’s, so the computer is struggling to line up its expectations with the image in front of it.
On regular people, it generally manages okay-ish, but it can accidentally slap a mustache where a shadow should be, or eyes in the frames of sunglasses that are hanging from shirts. Some people have even been able to get it to see ‘the man in the moon’ by using a low-resolution image of said moon! It doesn’t understand photos without people very well, so occasionally it’ll try to brute force one into existence and add a face where there are none, which led to some creepy pics of normal, but blurry, landscapes.
“Living” photos
Living photo apps allow you to bring photos ‘to life’. Some of them turn out really good! Some of them are downright uncanny. Apps that allow users to select specific features on their relative’s faces to animate appeared a few years ago, but it wasn’t really widespread til just recently. The picture would look around, blink, and appear to breathe as though it were a video, and it was fine if the portrait was taken at the standard ¾ or head on angles.
Any other angle, and the software couldn’t compensate for the tilt, and the face stretched or turned unnaturally, even with user input telling the software exactly where the eyes and mouth were. It had no way of knowing if a face was off-level, either, so the eyes blinked strangely diagonally even if the digital eyelids were lined up right, and the mouth would open at an odd angle to simulate breathing. Distinctive features such as an unusually wide or narrow face frequently got in the way, too, and having bits of the background move alongside the central face in the image could break the illusion.
The tech is great on the ideal picture, just not every picture.
Eraser Apps
Sometimes people get into the backgrounds of otherwise good photos. Sometimes flyaway hair ruins what should be a great shot, maybe there’s a plastic bag or something behind the subject of the photo. What can you do but hire a photo editor to remove these flaws? Now, you can recruit an AI to do it!
The AI works best on small things in non-cluttered backgrounds. Buildings may phase out of existence along with the thing that’s been erased, for example, because the AI is actually just sampling the surrounding image and putting it over the thing to be erased. Retouch, the one that got circulated around TikTok, does a brilliant job of erasing small problems on a plain background. Unfortunately, it can’t do that when the object is especially large, the background is especially cluttered (or has text in it), or people overlap the area to be erased.
It can only work off of information it’s given, so this pic makes some sort of sense, even if it came out bad.
The AI is programmed to look for certain elements to duplicate. It’s why the road still looks so coherent, but the girl is all sorts of messed up. It doesn’t have any arms to reference, and even if her other arm was visible, it wouldn’t mean much. It wouldn’t know that’s what it’s looking at or where it would need to place said arm, assuming it could recreate it for her other side. That still takes a human touch. Meanwhile, the straight line and consistent color patterns on the road are clear enough for it to recognize it as a road and continue the pattern.
This one, where an erased log was replaced with more of his shirt and a duplicate of the guy’s head, makes no sense! How did it decide his head belonged there? It had plenty of water to work off of, it just decided to sample the image up and to the right, contrary to what it did for the rest of the log! The shirt makes sense, but the head? Out of all of that water, the head is the part it duplicated? It’s sampling is pretty good – but not perfect!
Sources:
https://twitter.com/jonathanfly/status/1231079229279870976
https://screenrant.com/tiktok-bring-pictures-photos-to-life-myheritage-app-how/
