Automated Essay Scoring — Kaggle Competition End to End Project Implementation-Part 3

Source: Deep Learning on Medium

Automated Essay Scoring — Kaggle Competition End to End Project Implementation-Part 3

Kindly go through Part 1, Part 2 and Part 3 for complete understanding and project execution.

2. /mysite/grader/views.py for getting the context from web page and scoring the essay from saved model.

  • Django and utils libraries are imported where whenever user browse for web application first index function will be called which will internally call index.html file and load the same.
  • Essays definition will call essay.html file which will list all the essay where user or student can select which essay he/she wants to write.
  • Question definition takes the data from form in terms of content. Students written essay will get in terms of post request and store in the content.
content = form.cleaned_data.get('answer')
  • content has been converted into the testdataVectors using below code where Word2Vec saved model is used along with previously defined functions getAvgFeatureVec.
num_features = 300
model = word2vec.KeyedVectors.load_word2vec_format(os.path.join(current_path, "deep_learning_files/word2vec.bin"), binary=True)
clean_test_essays = []
clean_test_essays.append(essay_to_wordlist( content, remove_stopwords=True ))
testDataVecs = getAvgFeatureVecs( clean_test_essays, model, num_features )
testDataVecs = np.array(testDataVecs)
testDataVecs = np.reshape(testDataVecs, (testDataVecs.shape[0], 1, testDataVecs.shape[1]))
  • Now final_lstm.h5 model weights have been loaded and prediction is done in terms of score.
lstm_model = get_model()
lstm_model.load_weights(os.path.join(current_path, "deep_learning_files/final_lstm.h5"))
preds = lstm_model.predict(testDataVecs)
if math.isnan(preds):
preds = 0
else:
preds = np.around(preds)

if preds < 0:
preds = 0
if preds > question.max_score:
preds = question.max_score
else:
preds = 0
essay = Essay.objects.create(
content=content,
question=question,
score=preds
)
return redirect('essay', question_id=question.set, essay_id=essay.id)
else:
form = AnswerForm()

context = {
"question": question,
"form": form,
}
return render(request, 'grader/question.html', context)
  • Here predicted score has been passed back to question.html and score has been shown into the html page.

This is complete End to End Project Implementation of Automated Essay Scoring. Hip Hip Hurray !!!

Summary:

  1. You can try different neural network models as mentioned in the models folder and try to check different accuracy and see if you can increase the score.
  2. You can also use here pretrained models like GloVe, FastText or other state of art models as part of transfer learning which are currently in use.

3. If some of the code, you are not able to understand, then kindly Google it out for better understanding, by doing it your concepts will become much more clear.

4. There are 2 Research Paper mentioned which are good to go through so that you can get practise of it and they also gives clear picture of whole project.

Kindly go through Part 1 and Part 2 for complete understanding and project execution with given Github link.

References:

  1. Special thanks goes to Ronit Mankad’s Github Repo which I had forked and got end to end understanding about this project.
  2. References taken from Wikipedia and other websites.

If you really like this article series, kindly clap, follow me and enjoy the extreme power of artificial intelligence just like below minions.