Problem Creation for Jupyter Notebooks

Given below is the process through which you can create a problem of type: Data Science Jupyter Notebook.

• Click on the Add a Problem Button. This would open a Create a New Problem modal. In the problem type section, search for the Jupyter problem type. If the problem type is not available, then talk to your Customer Success Manager to get this type enabled in your account.

• Once you have filled in the basic details like Problem Name, Problem type, level of the question you are creating, you will be redirected to the Problem Creation Page as shown in the image below:

• You would be required to fill in the following generic fields:
  • Expected Solving Time
  • Description: You can be as descriptive as possible.
  • Difficulty Level
  • Scoring
  • Penalty
  • Discovery Tags/Insight Tags

• Next you will be asked to upload the datasets for this question type (.csv files with a maximum size of 50MB):
  • Training Dataset: This is the file which the candidate can use to train their models on. Should contain the target column for training.
  • Evaluation Dataset: This is the file which would be used to evaluate the performance of the candidate model. This file should also contain the target column, however this column would get hidden from the candidates when they read the file.
  • Validation Dataset: This is a file which can be used to show the format of the expected output file to the candidate.

• You can upload a .ipynb file which would get served to the candidate once they load up this question on the test UI. You can use this boilerplate to ask the candidate to install certain libraries, re arrange the output format etc.
  
  Images missing 
  
  
  • Next you will land on the Evaluation section. Here you would have to fill the following information:
    • Target Column: A dropdown from where you can select the column which contains the predictions.
    • ID Column: The column which contains the ID (serial number).
    • Problem Type: 
      • Binary Classification
      • Multi Class Classification Others (Regression)
    • Based on the problem type you have selected, you will get access to certain standard evaluation types which can aid the recruiter in decision making. Within a custom evaluation type, you would be required to add test cases for evaluation.
    • Positive Class: Defining the positive class within the target column for a binary evaluation problem.
  • After following the above steps you can proceed to add this question to the library. 

Congratulations, you have created your first Jupyter type question!