Week 15: Last Changes before Break

To test our product further, we looked in to questionnaires on mechanical turk and made plans to organize our own online survey.

Mechanical turk is a crowdsourcing marketplace where workers perform tasks (such as our questionnaire) and get paid per task/question. Because the testing of our tool takes about 30 to 40 minutes, it is more time and work than interviewees are willing to provide. Therefore, we want to give incentive to our interviewees to complete our tasks by posting it on mechanical turk (so they get paid per question).

After looking into a typical pay rate per question, we concluded that a payment of 15 cents per minute is to be expected. In order to shorten the online questionnaire, I ended up splitting it into three different ones, each testing a specific view of Build (list, pairwise, categorical). 

Putting in 15 cents per questions, where each questionnaire consists of 10 questions, we decided to limit our budget to 90 people. The below statistics show the approximate cost for mechanical turk.

With the end of the quarter, it was important to come up with a plan for next term. I constructed a week by week list of tasks that would need to be accomplished:

  • Finalize UI for Rankit
  • User Study 
    • online (mechanical turk)
    • onsite
  • Paper 


Popular posts from this blog

Week 31: Fair Ranking Search

Week 32: Datasets

Week 26: Finishing the Paper