Products

SurveyMonkey is built to handle every use case and need. Explore our product to learn how SurveyMonkey can work for you.

Get data-driven insights from a global leader in online surveys.

Explore core features and advanced tools in one powerful platform.

Build and customize online forms to collect info and payments.

Integrate with 100+ apps and plug-ins to get more done.

Purpose-built solutions for all of your market research needs.

Create better surveys and spot insights quickly with built-in AI.

Templates

Measure customer satisfaction and loyalty for your business.

Learn what makes customers happy and turn them into advocates.

Get actionable insights to improve the user experience.

Collect contact information from prospects, invitees, and more.

Easily collect and track RSVPs for your next event.

Find out what attendees want so that you can improve your next event.

Uncover insights to boost engagement and drive better results.

Get feedback from your attendees so you can run better meetings.

Use peer feedback to help improve employee performance.

Create better courses and improve teaching methods.

Learn how students rate the course material and its presentation.

Find out what your customers think about your new product ideas.

Resources

Best practices for using surveys and survey data

Our blog about surveys, tips for business, and more.

Tutorials and how to guides for using SurveyMonkey.

How top brands drive growth with SurveyMonkey.

Contact SalesLog in
Contact SalesLog in
Survey Science

What’s the best way to design a matrix question?

What’s the best way to design a matrix question?

Let’s put on our survey design caps, shall we?

When you want to ask many questions in a row that have the same response options, you’re most likely going to turn to the mighty matrix question.

This question format places similar questions in a single grid instead of presenting them one by one, like the following:

An example of a matrix question.

This type of question can help keep your survey short and easy to take, but it’s also easy to build one that's too long.

So what’s the right size of a matrix question? How many options is too many, and how should you organize matrix questions when you have, well, a lot of questions?

It’s good to limit your matrix questions to the minimum so as not to overwhelm yourself with the amount of data to analyze. It’s also important to think about your respondents’ experience when taking the survey. And don’t worry, we investigated just that in a recent study.

Matrix questions can help you collect more valuable responses—no matter the type of survey you run.

To really get data on the best way to design matrix questions, we ran an experiment on different matrix sizes to look at how the layout affected responses and participant experience.

We asked the same 20 questions on an agree-disagree scale to over 2,000 respondents via SurveyMonkey Audience. Some people had only 3 response options, some had 5, and some had 7.

Additionally, some saw questions with a maximum of 5 rows a page (meaning they had 4 pages for the 20 questions), some had 10 rows (2 pages with 10 questions per page), and some had 20 rows (all 20 questions on one page). By survey’s end, all respondents were asked to rate the survey’s layout and difficulty.

Some of the outcomes we looked at were:

  • Completion rate
  • Completion time
  • Ratings of the survey layout, difficulty
  • *Response consistency

Response consistency looks at how each person’s responses were consistent with their other responses.  In other words, if a person agrees with a statement at one point, they should generally agree with a similar one later, and disagree with an opposite of the statement.

Even though everyone had the same questions, the layout had a big effect on some of our outcomes. In particular, when there was a smaller matrix with only 5 rows on a page, as compared to larger matrix with 10 or 20 rows, people dropped out less and rated the survey as­ easier to take. There were no differences in reliability or completion time.

The results were clear—fewer is better! Even though they had to scroll through more pages, it was better and easier than having to look at a massive, 20-row matrix on a page, without any additional costs of time or consistency.

Graph of dropout percentage by matrix size

How about columns? This one will depend more on what your question is. In general, it’s a good idea to use columns at a minimum depending on your data. In this survey, people in the 7 column condition took longer to complete.

However, those in the 3 column version gave slightly less consistent responses, perhaps because they didn’t have as much room to give the responses that was more accurate for them.

Based on this study, we had a clear winner on the best size for a matrix question—5 rows by 5 columns.

Best matrix size of 5 rows by 5 columns

Now, this will of course depend on the exact number of questions you need and what kind of response options you have, but your best bet is again, go small.

Keep your matrix to no more than 5 rows per page, and your response options to the lowest necessary to get reasonable answers.

Most importantly, always think of the respondents’ experience when creating your survey. Put yourself in their shoes—would you want to wade through giant matrix questions while trying to complete a survey?

Get the highest quality data that you can and by designing your survey with these kinds of tips and more in mind will help you do just that.