You Can't Teach an Old Form New Tricks: Overfitting, Underfitting, Bias, and Variance

Some of the particulars that I found fascinating while attempting to coax responses from the machine satyr that I deem to be musically acceptable have been the use of ‘overfitting’ and ‘underfitting’48 techniques.

Overfitting happens when the neural network is very good at learning its training set, but cannot generalize beyond the training set (known as the generalization problem49 ). This results in a sort of mimicry in the system. The image below is a biased overfit model responding to the question: “if all of your friends were jumping off a bridge, would you?”

︎ Refer to Appendix A5 for ancillary musical, historical, and technical details

Overfitting is like trying to teach music by using a very specific training set. For example, listening or training only with madrigal chorale music. Someone who learns music only having listened to madrigal chorale music will learn a very specialized form of music and may not be able to stay in tune, timbre or idiom in another style. This model isn’t generalizable. Admittedly, most of the pieces in this project were overfit and deeply biased towards non-generalizable outcomes.

On the other hand I experienced ‘underfitting’ which happens when the network is not able to generate accurate predictions on the training set—not to mention the validation set. This ends up with some alien accompaniment that seemingly focuses on odd, myopic facets of data that has no correlation to general anthropometric semantics.

An example of underfitting could be the following: consider an AI satyr that tried to learn music solely from listening to Baroque lutenists, but ignored most of the phrasing, nuances, and general song structure and obsessed over micro-dynamics and timbre like the plucking of a string, trills, etc. This machine satyr will have an extremely limited understanding even of the notion of the Baroque musical forms, not to mention an insufficient ability to understand broader musical precepts.

48    "Model Fit: Underfitting vs. Overfitting - Amazon Machine Learning." https://docs.aws.amazon.com/machine-learning/latest/dg/model-fit-underfitting-vs-overfitting.html
49    "Generalization | Machine Learning | Google Developers." https://developers.google.com/machine-learning/crash-course/generalization/video-lecture