The problem has nothing to do with recursion. It seems the v8 and other JS engines only allow you to pass a certain number of arguments per function.
Math.max throws an error because each value has to be passed as an argument (exceeding the maximum), your custom max function works because it only takes one argument (the array of numbers).
Try to run the code using
Math.max in Firefox and will get an error message with a much better description:
Uncaught RangeError: too many function arguments