In addition to, or in combination with cross-validation, I recommend model blending.

Source: Deep Learning on Medium


In addition to, or in combination with cross-validation, I recommend model blending. The hidden decision trees methodology that I developed a while back, do just that:

  • decision trees on observations that are better handled by decision trees
  • regression on observations that are better handled by regression

Each model produces a score, and both scores (regression based or decision tree based) are re-scaled so that the two scores are interchangeable.

See details here: