Data Science and Prediction

Vasant Dhar states that data science or big data is gaining increasing significance with the potential of providing automated actionable knowledge creation and predictive models for use by both humans and computers. Data science implies a focus involving data and the systematic study of the organization, properties, and analysis of data and its role in inference, including confidence in the inference. Data science is different from statistics and other existing disciplines in several important ways. The emphasis on prediction is particularly strong in the machine learning and knowledge discovery in databases, or KDD, communities. The emphasis on predictive accuracy implicitly favors 'simple' theories over more complex theories in that the accuracy of sparser models tends to be more robust on future data. The requirement on predictive accuracy on observations that will occur in the future is a key consideration in data science.