Posts Tagged


What Do You Do When Search Engines Fail You?

Elizabeth Technology July 19, 2022

The internet is forever – finding your way back to things on it is not.

1) Try a Different Search Engine

It’s well known that Google’s mysterious SEO algorithms show different results based on the person searching, the time of day they searched, where they searched (phone or desktop? Etc.) and more. So, if Google’s not giving you results for this thing you want (or you’re getting a lot of ads instead of relevant results) try Bing, or DuckDuckGo. It might not help, but Bing’s less-labyrinthine SEO functions mean it’s not tripping over itself to correct a perceived typo or find a result that aligns with your star sign off of a vague search entry. Just because Google is the biggest doesn’t mean it’s always the best!

Alongside trying a different search engine, sometimes trying a different search phrase can help as well. Say you’re looking for a foreign film that someone mentioned to you, but you only remember that the title had ‘aytoils’ in it and it’s in French. If you search for that in English, Google is going to think you meant ‘atolls’, and the results are going to feature movies about atolls or tolls or atails even if you click ‘search for aytoils’, because Google doesn’t know what you meant and it’s trying to give you answers even if those answers suck and don’t match. However, if you translate your entire query into French, Google might guess you actually meant ‘étoiles’, the word for ‘star’ in French, and you’ll be just a smidge closer.

2) Forums

Ask for help from real people! If you’re looking for something hyperspecific, an active forum may be able to help where generic Google results do not. There are forums, large and small, all around the web, and those forums cover interests from slingshots to saltwater aquariums to pastry-making to anime. People generally want to share their hobbies, and people who are really into lizards or really into baking are more likely to have run into the same weird niche scenario you’re dealing with right now.

Of course, you should remember to be polite and follow forum rules if you go this route – you’re sourcing information directly from real people, and will often be conversing back and forth with them to get your answers. This is also one of the downsides to using a forum, as ‘real people’ includes beginners to intermediates in the craft, so think critically about the advice once you receive it and whether or not it makes sense for the thing you’re trying to achieve. It’s good to partner forum results with what you could gather from Google sources.

As an example where you should use caution, forums for breeding, trading, or buying and selling ball python snakes! Google will tell you that ball pythons with the ‘spider’ pattern on their back all have a condition called ‘wobble’. Wobble is present in every spider ball python and every derivative of it, but some snakes don’t get it so severely, leading the owner to believe their snake doesn’t have it or won’t pass it down. These people may give their individual snake anecdote to someone looking to buy a spider morph for breeding, while the first few results on Google definitively say every spider or morph with spider in it has wobble (and they do! But the severity is really unpredictable, so again, some people think their snake doesn’t have it or can’t pass it down.). They don’t mean to cause harm, and they most certainly didn’t tell someone to get a snake with this condition because they want snakes to suffer, but they don’t have complete information, and the disease is counterintuitive because you can’t breed it out.

 With all of this in mind, it’s still usually better than nothing, so check out a hobby forum if Google can only give you generic results!

2.5) Forums… But Also Answer Yourself

Cunningham’s Law states that people may not answer a question if left to their own devices, but they’ll be more likely to answer that question if they have the chance to correct someone who answered it wrong. You probably wouldn’t want to do this with anything serious or time sensitive (pets, illness, cars, what the effects of CO poisoning look like, etc.) but for minor things unlikely to result in serious property damage or personal injury, answering your own question about how hot-press watercolor paper holds up to gouache may attract an expert to correct you, especially on places like Reddit or TikTok where corrections get a lot of upvotes and views. It may also attract beginners (as noted above) but Cunningham’s Law applies to them, too.

3) Internet Archive Services

If you remember the URL or the website you first saw whatever you’re looking for, these services may be a ray of hope – they catalog what things used to look like on the web, just like a normal archiving service. They’re not completely perfect, they may have blind spots and gaps, and they’re definitely not first in most searches, but  if you’re getting desperate to see something the way it was, this might be a solution. Unfortunately, the death of Adobe Flash means a huge amount of Flash-reliant content (like games and some videos on sites like NewGrounds, Miniclip, and Nitrome) died with it as well, and even the internet archives can’t bring them back. While some people are working on projects to restore these games using browser plug-ins, it’s not looking good for the vast majority of them.  

4) Seek Out Accounts

‘Han Solo shot first’. You might have seen that sentiment online in Reddit arguments. In the original, unedited Star Wars trilogy, Han Solo fired on his business contact first, cementing his reputation as ruthlessly self-serving. In the Special Edition re-release, Greedo fires first, completely changing the context of Han shooting to kill him. His action is no longer morally gray; Greedo was going to kill him, what, was he not supposed to shoot in self-defense? People were rightfully mad at George Lucas for making this change, and many were even madder that the original film had just apparently vanished into thin air for the sake of pushing his Special Edition harder. But they remembered that Han fired first, and they never missed an opportunity to spread the word.

In some cases, seeking out accounts of how something was when you’ll never be able to find it can fill in a gap in research, if you can find people willing to share. Human memory is fallible, yes, but collectively, the old Star Wars fans knew ‘Han shot first’ and passed it down to their kids in a sort of oral history. Old TV shows, live-air bloopers, consumable products that expire and mold – sometimes, all you have of stuff that’s unsearchable is first- and secondhand accounts from the people who used them and remember them well enough to share with you. It’s not going to pop up in Google, but it might in a microfiche or interview!

Online Survey Questions: How to do it Wrong

Elizabeth Uncategorized October 4, 2021

Experts have struggled with this problem for decades. How do you measure something subjective, and how do you do it in a consistent way? How do you, a researcher, discover pain points and weak spots in your process for the customer? Carefully crafted questions with non-leading answers are difficult to make, though, so…

Here’s how to make a bad one.  

1) Make the question leading.

“How positively do you see Our Company”, “Do you think of Our Company in a positive light?” And “What is your opinion, on a scale of 1 to 5, of Our Company?” Are all going to get the company wildly different results. Using adjectives to describe the product or service to lead the customer into the ‘right’ answer is hardly new, but it’s always been a bad idea! Genuinely happy customers don’t need your survey questions to rail-road them into the right answer. Unhappy customers will be frustrated that they can’t make their opinion known. The data on the back end will suffer as a result.

Leading questions that express negative sentiments towards competitors and non-buyers are also likely to cause problems. “How unhappy were you with your previous company?” reads like a red flag – they might have switched for any number of reasons, and now they’re being forced to read these questions defensively. When someone breaks up with their partner, you don’t bad mouth the ex unless they start doing it first!

2) Don’t include any negative options.

If you don’t include a null option, you’re gonna get bad data. Picture a question asking you if you’ve heard of a handful of platforms, and think positively of them, and it assumes you’ve heard of at least one – that is, it doesn’t give you an option for ‘none of the above’.

This ad is a perfect example of what not to do! In a world with so much access to information, big companies are under the gun like never before. Between Google’s quiet removal of “Don’t Be Evil” from its mission statement, Apple’s use of child labor, Amazon’s poor working conditions, and Facebook’s data harvesting, none of these companies are particularly guilt-free. They’ve done bad things. Bad things that have a negative impact on the world. Their positive impacts are directly tied into the things that make them bad. There’s no ‘right’ answer where one is ‘good’ and the rest are ‘bad’.  The survey takers either select a company they haven’t heard bad news from lately, pick an answer they’re not happy with, or don’t select a company at all and move on without answering. None of the answers fit! Well-informed consumers are effectively excluded from the survey! Without a ‘none of the above’, a percent of potential surveyees can’t answer the question honestly.

While the initial results may hypothetically look like this:

The actual results could hypothetically be closer to this:

But the company has no way of knowing that ~25% of their customers, hypothetically, don’t like any of these options without a null option! Getting real numbers is critical for questions like these. Beating around the bush with “You must like one of them, right?” is only harming the companies themselves by giving them an inflated image of their public perception. People know they did bad things, this leading question isn’t going to make them forget that.

3) Only make the survey available after completing an action.

Asking users to sign up for your newsletter before granting them access to the survey will not only weed out the dispassionate users, it will also greatly annoy users who really want to leave constructive feedback!

Studies show that even getting customers to the survey (even if it’s digital!) is a great undertaking, usually reserved for people who are moderately happy or upset. When companies put barriers like newsletter signups in front of their users, they’re not getting willing sign-ups, they’re getting worse, more polarized results. Only very happy or very angry people are committed enough to sign up for the newsletter, and the inconvenience of having to do so may make both survey-takers slightly more negative in their answers. The people who were going to leave a 4/5 stars and a blank comment box are gone. That’s not a good thing. Instead of reinforcing valuable data, you’ve skewed your answers towards outliers!

If you really want users to sign up, include it as an option at the end of the form. Don’t force it – the engagement won’t be worth the irritation inflicted on the customer.  

4) Make your questions incredibly long and/or confusing.

Starting any 1-5 question with “Are you Aware of…?” “Do you…?” or “Have you considered…?” is just a bad time. Those are yes/no, survey-takers are going to be confused. Not every option is scalable! Poor wording can also make otherwise simple questions with simple responses a confusing word puzzle.

“Do you agree that contrary to popular belief, the general public should put sand into lakes and rivers?“

“Did our employees not do a good job?” “Was your ad experience intolerable?” “Do you agree that the job could or could not have been done better?”

None of these work with simple yes/no, and now the survey maker has to elaborate in the answers with a “yes, they should…” or “no, I didn’t…” which is more work than making the question straightforward.

On top of poor wording, making questions too long or too short for the customer to parse is a surefire way to confuse even the best readers! If you want bad results, make sentences go on forever when they don’t need to.

4) Let People Self-Select – and Then Call it Representative of the Whole

Former President Donald Trump was famous for running approval polls via newsletters and his favored conservative website ads. Obviously, people who don’t like him are not going to want to sign up for his newsletter or watch these forums, so they never saw these polls. People who did like him were more likely to be signed up to conservative sites and newsletters, and as a result, the survey results skewed massively conservative, and positive in his direction. People self-selected into news sources that he could exploit. In this way, he could claim that he was very positively viewed by the general public, when really the survey barely reached anyone outside the bubble. Yeah, sure, technically the survey was available on public websites… that doesn’t mean the public saw them. This is a public page, I could slap something together really quick and call it a survey, does that mean I ran a public survey with valid results? No.

The questions were also incredibly leading, designed to reinforce support for Trump and his political party instead of acting as questions, but we covered that in Question 1. No matter what you think of him, this is an objectively bad method for discovering data.

By citing these poorly-ran surveys, he was able to convince some that the media really was depicting him unfairly. After all, how could he have gotten an 80% approval rate out of his own survey when the media says he’s only at 30%? The answer: his organization picked outlets that loved him and called it an accurate representation of the real world, while the Gallup poll sampled randomly. Speaking of which…

5) Don’t Sample Randomly

It’s possible to avoid people self-selecting into your survey… and yet, you might still get a skewed answer out of them. It’s a lot of hard work to extract data from customers in a meaningful way. Random sampling is crucial!

If you want weird or minimal results, do something that keeps distinct portions of the population from answering. Make it difficult for customers on mobile to fill out the survey, and you’ll cut an enormous amount of potential participants out. Allow customer service agents to point out the end-of-experience survey only to the happy customers, and only get happy results. Pick certain zipcodes and ignore others for political polling, which will give any organization the answers it wants. Kick participants for having the ‘wrong’ results so you only get right ones.

Manage that, and you haven’t randomly sampled anything!

5.5) Give access to non-customers

Sometimes, however, you do need to sample within a population to get results for that population. I don’t own anything Apple-branded. If Apple were to survey me now about owning a product, I would clutter up their results with noise. And yet, Apple sometimes asks me what my opinion of their brand is in online adverts, without asking first if I’m ever going to buy an Apple product. I don’t care about Apple. My answer is noise. And yet, if I answer without maliciously picking 1s for everything, they won’t know that! They won’t dismiss me as an outlier. It’s okay to put a gate in front of the survey if it filters out noise and bogus survey responses.

6) Torture the data you do get

JD Powell’s company came into being when he watched his coworkers massage data over and over to tell their higher-ups what they wanted to hear: “Customers Are Happy and you’re doing a Great Job ™”. Of course, this didn’t reflect the reality of the situation, and the company continued to suffer from quality issues that could have been solved sooner if they’d listened. Why run a survey if you’re not actually looking for results? Why let customers think their voice is heard, only to cut some of them out for being a little too disagreeable? Outside survey companies exist because internal departments can’t show their managers the real, raw data – they shoot the messenger, and then wonder why profits are down if surveys are so good.

Every company wants to hear that customers are very happy. Every company wants that. Some companies want it so bad that they refuse to accept news of anything else. If you want to do a bad job, torture your data! You’ll have no idea what customers are actually thinking!

7) Ask everything, all at once

This is a bad idea. Interactive ad-surveys are supposed to be short and sweet. Ask too many questions and the customer may bail before they complete the survey. However, tracking one specific variable is for good surveys, and we’re not doing that. Do a bad job and ask every question, all at once. Get yourself three surveys worth of data.

One of the hardest parts of designing a survey is getting juuust enough information to build out your data. Jeep owners might be more likely to fish, that’s valuable data. How important is it to know if they own Apple products, when choosing what materials to make seat covers with? Will knowing the customer’s favorite TV show help your company determine what scent to make their detergent?

Data overload is really cool for finding weird correlations in things, but when the researcher has to present something with meaning to their higher-ups, it’s very easy to get lost in the sauce. A fifty-question survey is also far too intense for an ad, or for a post-shopping survey on a receipt, so that limits the pool to people signing up for surveys voluntarily instead of the hit-n-run kind found in ads.


Hyperlink for Formatting:

Buzzfeed Unsolved ARG: Was that a Good Idea?

Elizabeth Uncategorized August 2, 2021



ARG stands for alternate reality game, and I’ve written articles on them before. Essentially, it boils down to a sort of scavenger hunt that ends with a mystery being solved. Sometimes the mystery is unlocking game content, sometimes its discovering what happened to a character created for the ARG, sometimes its finding real treasure hidden by the game master. The sky’s the limit!

No matter what, though, they’re at their best when they have a large audience to brainstorm and solve them. ARGs try to bridge the gap between fiction and reality, so the more ‘real’ the first steps feel, the better the engagement, and the better the solutions from the audience. However…

There are times when you can get too real, or start in the wrong place for the product. An ARG run by one of Buzzfeed’s content creators got some negative feedback before it even really began.




Many of the fans in the comments were disgruntled not because of the content of the ARG, but rather where it was being shown – Aria Inthavong works as a presenter and content creator for Buzzfeed, and his focus has always been real cases. His show was the base for the ARG on the main channel. He has a second channel that’s more casual, and features YouTube livestreams alongside fan-based Q and A, and if anywhere was going to be the landing page for his ARG, I would have expected that second channel.

However, I suspect that Buzzfeed Unsolved Network is anticipating a content drought. One of its most popular series, featuring a pair of paranormal investigators exploring old and mysterious cases, is about to come to an end. A two-person show where both are good at improv (and the pair is clearly friends) is almost always going to be more engaging than a one-person show of the same format.

How do you fill a gap, especially one left by such a well-beloved part of the channel? You… sort of don’t. Buzzfeed brought on a bunch of one-person shows, and weeded out the weak, leaving only a handful as regulars on the channel. Aria Inthavong falls into this category, one of the strong remaining showrunners.


Buzzfeed Unsolved: Online


He’s only stayed so long because he’s good at what he does! Aria’s content focuses mainly on cases of internet tragedies, and he started just before the final season of Unsolved: Mysteries began airing. His stuff is digital: Youtubers who snap and go after other Youtubers, young folks who were persuaded by strangers online to kill their families or specific targets, harassment that escalated, video-taped murders posted to TikTok, etc. all sorts of things. All featuring death.

Aria tends to keep a very serious tone, occasionally stopping to add his own commentary to the script. Many of his cases are really ugly. Where Shane and Ryan spend a lot of time talking about cryptids, ghosts, and unattached people who have been dead for decades (or centuries), Aria’s cases are often very fresh, and relatives of the people in those cases may very well still be alive and watching. As such, the tone with which he handles cases is very important!

Unsolved: Online is consistently much more somber than the show that got Unsolved its running start.

Knowing that, it’s not so weird to see how fans are a little weirded out by the ARG’s introduction. It’s blending the lines a little too much for the fans’ liking. Imagine listening to a very real case about a very real murder in one video, and then in the next recommended video he’s asking for help finding a friend who sent videos with weird JPEG artifacts in them. You’d be a little weirded out, right? Or if you stumbled upon one of his mid-video cuts, one of the clues he’d left scattered along the way leading up to the ARG, you’d probably be more confused than anything. Obviously everything posted to that channel is heavily scrutinized by a team behind the scenes, there’s no way anything slipped by on accident. So why even begin with the pretense that this may be real?


The Content of the ARG


Of course, if you see the ‘official ARG starts now’ video with no context, you’re likely to realize pretty quickly that it’s a game. Despite what some say, that’s a good thing! You don’t want well-meaning kids contacting the police over your Youtube video, after all. However, the logic of an ARG means you have to stick to the script that it’s all real, even if you find people pointing out logical inconsistencies to you in the comments. You’d never post a video about a friend going missing without doing anything else to help first, right? Especially when you’re constantly reviewing cases where people didn’t and then bodies were recovered later as a result? The first stepping stone was a little rough, and for more than just placement.

Anyway, the content as of this article is just heating up. Aria’s ARG consists of a couple of mysterious videos and text messages that his friend sent him before disappearing off the face of the Earth. Strange, face-like glitches appear in the videos he shows to the audience, which include a video of his friend’s livestream where he melts into RGB fuzz before said stream cuts out. He looks tired, like he’s been trying to decipher the messages himself overnight, and he reads off messages that don’t sound like his friend. He projects a certain level of frantic uncertainty about the whole thing. Essentially, Aria Inthavong asks his fans for help deciphering his friend’s final messages to find him.

Before that, he also included little hints that something was going to happen soon. He’d disappear from his chair for a second in the middle of a sentence. He’d pause, and look behind him for movement. It would suddenly cut to him facing the wrong way before jerking back to his original statement, or just continuing as if nothing happened.

He then uploads the supposed glitched videos, and plugs his Twitter as a place to post what fans have ‘discovered’ and put together clues as a community. It’s obviously an ARG. Obviously.




Like I said earlier, placement could have been better. Showing faked videos alongside real ones is a little odd for a channel that has otherwise claimed to focus on the truth of the matters it covers. Healthy skepticism is always present in the ghost videos, every option and every suspect are covered when it comes to murder cases, even boring ones that would rule out more “cool” and “mysterious” stories. For an infotainment channel, that is actually saying a lot – other channels that claim to investigate things tend to throw their hands up and say “yep, must have been a ghost! Must have been aliens! Must have been Bigfoot!”

On the surface, it looks like Buzzfeed Unsolved Network is abusing the trust they’ve built with their community to add an extra air of mystery and trustworthiness to the ARG. I don’t think that’s what they intended. As previously said, you’d never want someone to misinterpret your plea for help solving puzzles as a genuine plea for help finding a missing person. Besides, it’s been pretty obvious that something was in the works for a while now, with weird gaps in videos and strange artifacts left in content that could have easily been removed. Nobody could reasonably misinterpret that as real except for children, and that channel isn’t aimed at children.

I think at any other location, this would have gone over very well. Aria is well-liked, after all, and he has a small fanbase all his own. It would be foolish to believe that was Buzzfeed attempting to pass this haunting off as real. But intent and reception are two very different things!