Machine Learning and Big Data

This book is intended for academic and industrial developers, exploring and developing applications in the area of big data and machine learning, including those that are solving technology requirements, evaluation of methodology advances and algorithm demonstrations.

The intent of this book is to provide awareness of algorithms used for machine learning and big data in the academic and professional community. The 17 chapters are divided into 5 sections: Theoretical Fundamentals; Big Data and Pattern Recognition; Machine Learning: Algorithms & Applications; Machine Learning's Next Frontier and Hands-On and Case Study. While it dwells on the foundations of machine learning and big data as a part of analytics, it also focuses on contemporary topics for research and development. In this regard, the book covers machine learning algorithms and their modern applications in developing automated systems.

Subjects covered in detail include:

• Mathematical foundations of machine learning with various examples.

• An empirical study of supervised learning algorithms like Naïve Bayes, KNN and semi-supervised learning algorithms viz. S3VM, Graph-Based, Multiview.

• Precise study on unsupervised learning algorithms like GMM, K-mean clustering, Dritchlet process mixture model, X-means and Reinforcement learning algorithm with Q learning, R learning, TD learning, SARSA Learning, and so forth.

• Hands-on machine leaning open source tools viz. Apache Mahout, H2O.

• Case studies for readers to analyze the prescribed cases and present their solutions or interpretations with intrusion detection in MANETS using machine learning.

• Showcase on novel user-cases: Implications of Electronic Governance as well as Pragmatic Study of BD/ML technologies for agriculture, healthcare, social media, industry, banking, insurance and so on.

Auteur

Uma N. Dulhare is a Professor in the Department of Computer Science & Eng., MJCET affiliated to Osmania University, Hyderabad, India. She has more than 20 years teaching experience years with many publications in reputed international conferences, journals and online book chapter contributions. She received her PhD from Osmania University, Hyderabad. Khaleel Ahmad is an Assistant Professor in the Department of Computer Science & Information Technology at Maulana Azad National Urdu University, Hyderabad, India. He holds a PhD in Computer Science & Engineering. He has published more than 25 papers in refereed journals and conferences as well as edited two books. Khairol Amali bin Ahmad obtained a BSc in Electrical Engineering in 1992 from the United States Military Academy, West Point, MSc in Military Electronic Systems Engineering in 1999 from Cranfield University, England, and PhD from ISAE-SUPAERO, France in 2015. Currently, he is the Dean of the Engineering Faculty at the National Defense University of Malaysia.

Contenu
Preface xix

Section 1: Theoretical Fundamentals 1

1 Mathematical Foundation 3
*Afroz and Basharat Hussain*

1.1 Concept of Linear Algebra 3

1.1.1 Introduction 3

1.1.2 Vector Spaces 5

1.1.3 Linear Combination 6

1.1.4 Linearly Dependent and Independent Vectors 7

1.1.5 Linear Span, Basis and Subspace 8

1.1.6 Linear Transformation (or Linear Map) 9

1.1.7 Matrix Representation of Linear Transformation 10

1.1.8 Range and Null Space of Linear Transformation 13

1.1.9 Invertible Linear Transformation 15

1.2 Eigenvalues, Eigenvectors, and Eigendecomposition of a Matrix 15

1.2.1 Characteristics Polynomial 16

1.2.1.1 Some Results on Eigenvalue 16

1.2.2 Eigendecomposition 18

1.3 Introduction to Calculus 20

1.3.1 Function 20

1.3.2 Limits of Functions 21

1.3.2.1 Some Properties of Limits 22

1.3.2.2 1nfinite Limits 25

1.3.2.3 Limits at Infinity 26

1.3.3 Continuous Functions and Discontinuous Functions 26

1.3.3.1 Discontinuous Functions 27

1.3.3.2 Properties of Continuous Function 27

1.3.4 Differentiation 28

References 29

2 Theory of Probability 31
*Parvaze Ahmad Dar and Afroz*

2.1 Introduction 31

2.1.1 Definition 31

2.1.1.1 Statistical Definition of Probability 31

2.1.1.2 Mathematical Definition of Probability 32

2.1.2 Some Basic Terms of Probability 32

2.1.2.1 Trial and Event 32

2.1.2.2 Exhaustive Events (Exhaustive Cases) 33

2.1.2.3 Mutually Exclusive Events 33

2.1.2.4 Equally Likely Events 33

2.1.2.5 Certain Event or Sure Event 33

2.1.2.6 Impossible Event or Null Event () 33

2.1.2.7 Sample Space 34

2.1.2.8 Permutation and Combination 34

2.1.2.9 Examples 35

2.2 Independence in Probability 38

2.2.1 Independent Events 38

2.2.2 Examples: Solve the Following Problems 38

2.3 Conditional Probability 41

2.3.1 Definition 41

2.3.2 Mutually Independent Events 42

2.3.3 Examples 42

2.4 Cumulative Distribution Function 43

2.4.1 Properties 44

2.4.2 Example 44

2.5 Baye's Theorem 46

2.5.1 Theorem 46

2.5.1.1 Examples 47

2.6 Multivariate Gaussian Function 50

2.6.1 Definition 50

2.6.1.1 Univariate Gaussian (i.e., One Variable Gaussian) 50

2.6.1.2 Degenerate Univariate Gaussian 51

2.6.1.3 Multivariate Gaussian 51

References 51

3 Correlation and Regression 53
*Mohd. Abdul Haleem Rizwan*

3.1 Introduction 53

3.2 Correlation 54

3.2.1 Positive Correlation and Negative Correlation 54

3.2.2 Simple Correlation and Multiple Correlation 54

3.2.3 Partial Correlation and Total Correlation 54

3.2.4 Correlation Coefficient 55

3.3 Regression 57

3.3.1 Linear Regression 64

3.3.2 Logistic Regression 64

3.3.3 Polynomial Regression 65

3.3.4 Stepwise Regression 66

3.3.5 Ridge Regression 67

3.3.6 Lasso Regression 67

3.3.7 Elastic Net Regression 68

3.4 Conclusion 68

References 69

Section 2: Big Data and Pattern Recognition 71

4 Data Preprocess 73
*Md. Sharif Hossen*

4.1 Introduction 73

4.1.1 Need of Data Preprocessing 74

4.1.2 Main Tasks in Data Preprocessing 75

4.2 Data Cleaning 77

4.2.1 Missing Data 77

4.2.2 Noisy Data 78

4.3 Data Integration 80

4.3.1 2 Correlation Test 82

4.3.2 Correlation Coecient Test 82

4.3.3 Covariance Test 83 <p&g...