Week 2: Finalizing Plans

We knew we had a tool. Naturally, the next question was, how do we incorporate it in our project?

Planing Landing Page 

The initial thoughts on the landing page lead us to determine that we have three tools: Lineup, MyRanker, Data. While Lineup would serve as a visualization tool, MyRanker would be the machine learning tool that decides how to rank based on a sample of user preferences, and Data would be a tool to store and manage user related data (their data files, which have different attributes and their values). I worked with another teammate to mockup our landing page, which would act as the index page that provides access to the three tools.

Database Storage

After a talk with our graduate student advisor, we've agreed that we did not want to get stuck on data storage and management, and that it would be best to minimize the project of the Data tool. It would only support importing and exporting user datasets. Therefore, no data management, such as adding or modifying specific attributes and their values would be supported. 

So we modified our basic model.

While weighting the pros and cons of one database over another, we ended up with two choices: MongoDB or Firebase. 

Researching MongoDB came up with a couple of difficulties we would encounter.

  • Figure out how to encrypt passwords
    • Solution: bcrypt
      • Password hashing function 
      • Package that can be installed on npm
  • How to authenticate users
    • Encrypting our communication with the server 
    • Avoid attackers from stealing identity of legitimate database users
    • Defense against brute force attacks
      • Keep track of login attempts
      • IP verification
  • Default configuration for MongoDB allow everyone full access to the database
    • Don't want users to have access to other user's data
  • Prevent bot from signing in 
    • Solution: ReCaptcha
  • Secure database
    • Prevent against SQL injections
  • How do we backup data?
And, while MongoDB allowed for lots of storage for free, Firebase provided solutions to the above problems. As long as our number of users was minimal, we benefited a lot from Firebase (since higher number of user would cost money to maintain).

Firebase data will be stored on the cloud (so no problems with backup), authentication would be handled by popular providers (such as Google, Facebook, Twitter, Github), communication with our database would be secure, no need to fear attacks on database since it's handled by Firebase.

Therefore, we decided to move the Data tool to the cloud:

Decoding Lineup 

My team and I gathered for a meeting that lasted four hours straight. Our task was to figure out how Lineup works and finding out a place where we could engineer a communication feature that would to talk to our too, MyRanker. It would have been time inefficient for all of us to focus on the exact implementation of Lineup. To combat getting stuck on one file for too long, we devised a plan for each of us to pick a file, spend 10 minutes to understand the functionality and importance of the file, and move on to the next set of files. 

Our discovery led to a couple of findings.

Bad news: Lineup is a ranking (bargraph) visualization tool build on top of a data visualization tool Phovea. So there was a whole another chunk of code that we needed to look at in order to understand Lineup.

Good news: There exists a special column called "Combined" column that accepts a json file with all the data and weights for each attribute. Therefore, if MyRanker can figure out the weights, we can just send it in a form of a json combined column to Lineup for visualization. So, there was no need to figure out more of the Lineup and/or Phovea code.


Popular posts from this blog

Week 31: Fair Ranking Search

Week 32: Datasets

Week 26: Finishing the Paper