I upgraded to 1.0.1 last evening.
I’ve been working with eval_metric = ‘rmsle’ and objective = ‘reg:squaredlogerror’.
I’ve also been working with the Python equivalents from the custom metric and objective tutorial.
I encountered some issues with the internal version of rmsle. I tried to reproduce them using the Python equivalent and got some surprising (to me) results.
Here they are. The Python version returns values but the internal version returns mostly nans.
I tried this last evening without using the Python equivalents and still got nans from the internal objective and metric.
[0] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.14098 training-PyRMSLE:1.04673 validation-PyRMSLE:1.05030 recents-PyRMSLE:0.14098
[1] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.09377 training-PyRMSLE:1.04167 validation-PyRMSLE:1.04507 recents-PyRMSLE:0.09377
[2] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.08088 training-PyRMSLE:1.03936 validation-PyRMSLE:1.04262 recents-PyRMSLE:0.08088
[3] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.07554 training-PyRMSLE:1.03808 validation-PyRMSLE:1.04125 recents-PyRMSLE:0.07554
[4] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.07433 training-PyRMSLE:1.03732 validation-PyRMSLE:1.04042 recents-PyRMSLE:0.07433
[5] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.07352 training-PyRMSLE:1.03687 validation-PyRMSLE:1.03993 recents-PyRMSLE:0.07352
[6] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.07167 training-PyRMSLE:1.03660 validation-PyRMSLE:1.03963 recents-PyRMSLE:0.07167
[7] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.07164 training-PyRMSLE:1.03645 validation-PyRMSLE:1.03946 recents-PyRMSLE:0.07164
[8] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06925 training-PyRMSLE:1.03635 validation-PyRMSLE:1.03937 recents-PyRMSLE:0.06925
[9] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06949 training-PyRMSLE:1.03631 validation-PyRMSLE:1.03932 recents-PyRMSLE:0.06949
[10] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06945 training-PyRMSLE:1.03628 validation-PyRMSLE:1.03929 recents-PyRMSLE:0.06945
[11] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06916 training-PyRMSLE:1.03626 validation-PyRMSLE:1.03927 recents-PyRMSLE:0.06916
[12] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06919 training-PyRMSLE:1.03625 validation-PyRMSLE:1.03926 recents-PyRMSLE:0.06919
[13] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06929 training-PyRMSLE:1.03624 validation-PyRMSLE:1.03925 recents-PyRMSLE:0.06929
[14] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06774 training-PyRMSLE:1.03623 validation-PyRMSLE:1.03924 recents-PyRMSLE:0.06774
[15] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06753 training-PyRMSLE:1.03623 validation-PyRMSLE:1.03923 recents-PyRMSLE:0.06753
[16] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06733 training-PyRMSLE:1.03622 validation-PyRMSLE:1.03923 recents-PyRMSLE:0.06733
[17] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06756 training-PyRMSLE:1.03622 validation-PyRMSLE:1.03923 recents-PyRMSLE:0.06756
[18] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06757 training-PyRMSLE:1.03621 validation-PyRMSLE:1.03922 recents-PyRMSLE:0.06757
[19] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06650 training-PyRMSLE:1.03621 validation-PyRMSLE:1.03922 recents-PyRMSLE:0.06650
[20] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06623 training-PyRMSLE:1.03621 validation-PyRMSLE:1.03921 recents-PyRMSLE:0.06623
[21] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06684 training-PyRMSLE:1.03620 validation-PyRMSLE:1.03921 recents-PyRMSLE:0.06684
[22] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06689 training-PyRMSLE:1.03620 validation-PyRMSLE:1.03921 recents-PyRMSLE:0.06689
[23] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06675 training-PyRMSLE:1.03620 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06675
[24] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06588 training-PyRMSLE:1.03620 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06588
[25] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06635 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06635
[26] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06635 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06635
[27] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06646 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06646
[28] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06648 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06648
[29] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06648 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06648
[30] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06619 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06619
[31] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06616 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03920 recents-PyRMSLE:0.06616
[32] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[33] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03619 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[34] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[35] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[36] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[37] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[38] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[39] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[40] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[41] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[42] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[43] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[44] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[45] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[46] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[47] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[48] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[49] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[50] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[51] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06561 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[52] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06561
[53] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[54] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[55] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[56] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[57] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[58] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[59] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[60] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[61] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[62] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[63] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[64] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[65] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[66] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[67] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[68] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[69] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[70] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[71] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[72] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[73] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[74] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[75] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[76] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[77] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[78] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[79] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[80] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[81] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[82] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[83] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[84] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[85] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[86] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[87] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[88] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[89] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[90] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[91] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[92] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[93] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[94] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[95] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[96] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[97] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[98] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562
[99] training-rmsle:nan validation-rmsle:nan recents-rmsle:0.06562 training-PyRMSLE:1.03618 validation-PyRMSLE:1.03919 recents-PyRMSLE:0.06562