Market researchers know you more than you know yourself: Market research has changed drastically from the days of people knocking on doors with a clipboard asking for a few minutes of your time. Today, market research can tap into our every move on the internet: From what we search, what we click on, what we buy, to how long we stay on each site. With no mortal capable of making sense of all this data, technology has come to the rescue, offering artificial intelligence (AI) and data processing to interpret human behavior, better than humans themselves can.

People don’t always know why they’re attracted to things, but AI does: Companies such as Colourtext track patterns among internet users to determine how things or people are truly perceived, writes the BBC, helping them break down or defy traditional categories that the market research industry uses to define our taste. For example, by analyzing the readership of major UK news sites, they found that Boris Johnson was perceived as more of a celebrity than a politician (sorry Boris).

Even your face is a potential data trove: The advent of software that analyzes facial expressions has added even more to market researchers’ toolkits, enabling the detection of subconscious feelings alongside spoken answers. Even if someone says they don’t like a rom-com (because it’s embarrassing), the AI will be able to detect the authenticity of their statement based on facial cues. The software can even take into account the age, gender, and race of the person it’s analyzing to give an accurate study of what they’re feeling.

How does it work? When the US fast food restaurant chain Del Taco changed its decor and menu and wanted to get a sense of the customer’s opinion, they tapped US analytics software house Medallia to use this kind of technology. Medallia’s team stepped in with a survey app and asked restaurant goers to answer questions via video on their smartphones. The data from the videos were all streamed into the cloud for analysis and used to expose underlying emotions about the food they were eating by looking at their language, tone of voice, and facial expressions as they spoke. The technique gives a “much richer view than a traditional survey,” believes Carl Wong, an industry insider.

Traditionally, people may not have answered surveys very truthfully: “Talking to a camera is a very effective tool, but what people say is often miles apart from what they feel, there are so many unconscious factors at play,” Jon Puleston, who analyses online material for consumer research giant Kantar, told the BBC. AI can help researchers bypass embarrassment, reluctance, or unconscious biases that may affect how a participant responds to survey questions, by revealing more about what they think from their facial expressions.

Video surveys are just the tip of the iceberg: Social media analysis, sentiment analysis, consumer engagement monitoring, heat mapping, and quick polls, are just some of the tools now leveraged by researchers. With all these options available, companies are opting for ultra-specific analyses to answer precarious business questions as quickly as possible, according to an op-ed in Marketing Interactive. And with covid-19 causing rapid changes to consumer habits, marketeers now require more agile research methods to keep up with the sudden changes in business environments.

But human intelligence isn’t completely irrelevant just yet: Qualtrics echoes the need for agility, but maintains that a blended approach to analyzing information that utilizes human intelligence and creativity is the most effective method of creating and communicating ideas in a compelling way.