Recommender System with Machine Learning and Artificial Intelligence

Sachi Nandan Mohanty received his PhD from IIT Kharagpur, India in 2015 and is now at ICFAI Foundation for Higher Education, Hyderabad, India. Jyotir Moy Chatterjee is working as an Assistant Professor (IT) at Lord Buddha Education Foundation, Kathmandu, Nepal. He has completed M.Tech in Computer Science & Engineering from Kalinga Institute of Industrial Technology, Bhubaneswar, India. Sarika Jain obtained her PhD in the field of Knowledge Representation in Artificial Intelligence in 2011. She has served in the field of education for over 18 years and is currently in service at the National Institute of Technology, Kurukshetra. Ahmed A. Elngar is the Founder and Head of Scientific Innovation Research Group (SIRG) and Assistant Professor of Computer Science at the Faculty of Computers and Information, Beni-Suef University, Egypt. Priya Gupta is working as an Assistant Professor in the Department of Computer Science at Maharaja Agrasen College, University of Delhi. Her Doctoral Degree is from BIT (Mesra), Ranchi.

Preface xix

Acknowledgment xxiii

Part 1: Introduction to Recommender Systems 1

1 An Introduction to Basic Concepts on Recommender Systems 3
Pooja Rana, Nishi Jain and Usha Mittal

1.1 Introduction 4

1.2 Functions of Recommendation Systems 5

1.3 Data and Knowledge Sources 6

1.4 Types of Recommendation Systems 8

1.4.1 Content-Based 8

1.4.1.1 Advantages of Content-Based Recommendation 11

1.4.1.2 Disadvantages of Content-Based Recommendation 11

1.4.2 Collaborative Filtering 12

1.5 Item-Based Recommendation vs. User-Based Recommendation System 14

1.5.1 Advantages of Memory-Based Collaborative Filtering 15

1.5.2 Shortcomings 16

1.5.3 Advantages of Model-Based Collaborative Filtering 17

1.5.4 Shortcomings 17

1.5.5 Hybrid Recommendation System 17

1.5.6 Advantages of Hybrid Recommendation Systems 18

1.5.7 Shortcomings 18

1.5.8 Other Recommendation Systems 18

1.6 Evaluation Metrics for Recommendation Engines 19

1.7 Problems with Recommendation Systems and Possible Solutions 20

1.7.1 Advantages of Recommendation Systems 23

1.7.2 Disadvantages of Recommendation Systems 24

1.8 Applications of Recommender Systems 24

References 25

2 A Brief Model Overview of Personalized Recommendation to Citizens in the Health-Care Industry 27
Subhasish Mohapatra and Kunal Anand

2.1 Introduction 28

2.2 Methods Used in Recommender System 29

2.2.1 Content-Based 29

2.2.2 Collaborative Filtering 32

2.2.3 Hybrid Filtering 33

2.3 Related Work 33

2.4 Types of Explanation 34

2.5 Explanation Methodology 35

2.5.1 Collaborative-Based 36

2.5.2 Content-Based 36

2.5.3 Knowledge and Utility-Based 37

2.5.4 Case-Based 37

2.5.5 Demographic-Based 38

2.6 Proposed Theoretical Framework for Explanation-Based Recommender System in Health-Care Domain 39

2.7 Flowchart 39

2.8 Conclusion 41

References 41

3 2Es of TIS: A Review of Information Exchange and Extraction in Tourism Information Systems 45
Malik M. Saad Missen, Mickaël Coustaty, Hina Asmat, Amnah Firdous, Nadeem Akhtar, Muhammad Akram and V. B. Surya Prasath

3.1 Introduction 46

3.2 Information Exchange 49

3.2.1 Exchange of Tourism Objects Data 49

3.2.1.1 Semantic Clashes 50

3.2.1.2 Structural Clashes 50

3.2.2 Schema.org—The Future 51

3.2.2.1 Schema.org Extension Mechanism 52

3.2.2.2 Schema.org Tourism Vocabulary 52

3.2.3 Exchange of Tourism-Related Statistical Data 53

3.3 Information Extraction 55

3.3.1 Opinion Extraction 56

3.3.2 Opinion Mining 57

3.4 Sentiment Annotation 57

3.4.1 SentiML 58

3.4.1.1 SentiML Example 58

3.4.2 OpinionMiningML 59

3.4.2.1 OpinionMiningML Example 60

3.4.3 EmotionML 61

3.4.3.1 EmotionML Example 61

3.5 Comparison of Different Annotations Schemes 62

3.6 Temporal and Event Extraction 64

3.7 TimeML 65

3.8 Conclusions 67

References 67

Part 2: Machine Learning-Based Recommender Systems 71

4 Concepts of Recommendation System from the Perspective of Machine Learning 73
Sumanta Chandra Mishra Sharma, Adway Mitra and Deepayan Chakraborty

4.1 Introduction 73

4.2 Entities of Recommendation System 74

4.2.1 User 74

4.2.2 Items 75

4.2.3 Action 75

4.3 Techniques of Recommendation 76

4.3.1 Personalized Recommendation System 77

4.3.2 Non-Personalized Recommendation System 77

4.3.3 Content-Based Filtering 77

4.3.4 Collaborative Filtering 78

4.3.5 Model-Based Filtering 80

4.3.6 Memory-Based Filtering 80

4.3.7 Hybrid Recommendation Technique 81

4.3.8 Social Media Recommendation Technique 82

4.4 Performance Evaluation 82

4.5 Challenges 83

4.5.1 Sparsity of Data 84

4.5.2 Scalability 84

4.5.3 Slow Start 84

4.5.4 Gray Sheep and Black Sheep 84

4.5.5 Item Duplication 84

4.5.6 Privacy Issue 84

4.5.7 Biasness 85

4.6 Applications 85

4.7 Conclusion 85

References 85

5 A Machine Learning Approach to Recommend Suitable Crops and Fertilizers for Agriculture 89
Govind Kumar Jha, Preetish Ranjan and Manish Gaur

5.1 Introduction 90

5.2 Literature Review 91

5.3 Methodology 93

5.4 Results and Analysis 96

5.5 Conclusion 97

References 98

6 Accuracy-Assured Privacy-Preserving Recommender System Using Hybrid-Based Deep Learning Method 101
Abhaya Kumar Sahoo and Chittaranjan Pradhan

6.1 Introduction 102

6.2 Overview of Recommender System 103

6.3 Collaborative Filtering-Based Recommender System 106

6.4 Machine Learning Methods Used in Recommender System 107

6.5 Proposed RBM Model-Based Movie Recommender System 110

6.6 Proposed CRBM Model-Based Movie Recommender System 113

6.7 Conclusion and Future Work 115

References 118

7 Machine Learning-Based Recommender System for Breast Cancer Prognosis 121
G. Kanimozhi, P. Shanmugavadivu and M. Mary Shanthi Rani

7.1 Introduction 122

7.2 Related Works 124

7.3 Methodology 125

7.3.1 Experimental Dataset 125

7.3.2 Feature Selection 127

7.3.3 Functional Phases of MLRS-BC 128

7.3.4 Prediction Algorithms 129

7.4 Results and Discussion 131

7.5 Conclusion 138

Acknowledgment 139

References 139

8 A Recommended System for Crop Disease Detection and Yield Prediction Using Machine Learning Approach 141
Pooja Akulwar

8.1 Introduction 142

8.2 Machine Learning 143

8.2.1 Overview 143

8.2.2 Machine Learning Algorithms 145

8.2.3 Machine Learning Methods 146

8.2.3.1 Artificial Neural Network 146

8.2.3.2 Support Vector Machines 146

8.2.3.3 K-Nearest Neighbors (K-NN) 147

8.2.3.4 Decision Tree Learning 147

8.2.3.5 Random Forest 148

8.2.3.6 Gradient Boosted Decision Tree (GBDT) 149

8.2.3.7 Regularized Greedy Forest (RGF) 150

8.3 Recommender System 151

8.3.1 Overview 151

8.4 Crop Management 153

8.4.1 Yield Prediction 153

8.4.2 Disease Detection 154

8.4.3 Weed Detection 156

8.4.4 Crop Quality 159

8.5 Application—Crop Disease Detection and Yield Prediction 159

References 162

Part 3: Content-Based Recommender Systems 165

9 Content-Based Recommender Systems 167
Poonam Bhatia Anand and Rajender Nath

9.1 Introduction 167

9.2 Literature Review 168

9.3 Recommendation Process 172

9.3.1 Architecture of Content-Based Recommender System 172

9.3.2 Profile Cleaner Representation 175

9.4 Techniques Used for Item Representation and Learning User Profile 176

9.4.1 Representation of Content 176

9.4.2 Vector Space Model Based on Keywords 177

9.4.3 Techniques for Learning Profiles of User 179

9.4.3.1 Probabilistic Method 179

9.4.3.2 Rocchio’s and Relevance Feedback Method 180

9.4.3.3 Other Methods 181

9.5 Applicability of Recommender System in Healthcare and Agriculture 182

9.5.1 Recommendation System in Healthcare 182

9.5.2 Recommender System in Agriculture 184

9.6 Pros and Cons of Content-Based Recommender System 186

9.7 Conclusion 187

References 188

10 Content (Item)-Based Recommendation System 197
R. Balamurali

10.1 Introduction 198

10.2 Phases of Content-Based Recommendation Generation 198

10.3 Content-Based Recommendation Using Cosine Similarity 199

10.4 Content-Based Recommendations Using Optimization Techniques 204

10.5 Content-Based Recommendation Using the Tree Induction Algorithm 208

10.6 Summary 212

References 213

11 Content-Based Health Recommender Systems 215
Soumya Prakash Rana, Maitreyee Dey, Javier Prieto and Sandra Dudley

11.1 Introduction 216

11.2 Typical Health Recommender System Framework 217

11.3 Components of Content-Based Health Recommender System 218

11.4 Unstructured Data Processing 220

11.5 Unsupervised Feature Extraction & Weighting 221

11.5.1 Bag of Words (BoW) 221

11.5.2 Word to Vector (Word2Vec) 222

11.5.3 Global Vectors for Word Representations (Glove) 222

11.6 Supervised Feature Selection & Weighting 222

11.7 Feedback Collection 225

11.7.1 Medication & Therapy 225

11.7.2 Healthy Diet Plan 225

11.7.3 Suggestions 225

11.8 Training & Health Recommendation Generation 226

11.8.1 Analogy-Based ML in CBHRS 227

11.8.2 Specimen-Based ML in CBHRS 227

11.9 Evaluation of Content Based Health Recommender System 228

11.10 Design Criteria of CBHRS 229

11.10.1 Micro-Level & Lucidity 230

11.10.2 Interactive Interface 230

11.10.3 Data Protection 230

11.10.4 Risk & Uncertainty Management 231

11.10.5 Doctor-in-Loop (DiL) 231

11.11 Conclusions and Future Research Directions 231

References 233

12 Context-Based Social Media Recommendation System 237
R. Sujithra Kanmani and B. Surendiran

12.1 Introduction 237

12.2 Literature Survey 240

12.3 Motivation and Objectives 241

12.3.1 Architecture 241

12.3.2 Modules 242

12.3.3 Implementation Details 243

12.4 Performance Measures 243

12.5 Precision 243

12.6 Recall 243

12.7 F- Measure 244

12.8 Evaluation Results 244

12.9 Conclusion and Future Work 247

References 248

13 Netflix Challenge—Improving Movie Recommendations 251
Vasu Goel

13.1 Introduction 251

13.2 Data Preprocessing 252

13.3 MovieLens Data 253

13.4 Data Exploration 255

13.5 Distributions 256

13.6 Data Analysis 257

13.7 Results 265

13.8 Conclusion 266

References 266

14 Product or Item-Based Recommender System 269
Jyoti Rani, Usha Mittal and Geetika Gupta

14.1 Introduction 270

14.2 Various Techniques to Design Food Recommendation System 271

14.2.1 Collaborative Filtering Recommender Systems 271

14.2.2 Content-Based Recommender Systems (CB) 272

14.2.3 Knowledge-Based Recommender Systems 272

14.2.4 Hybrid Recommender Systems 273

14.2.5 Context Aware Approaches 273

14.2.6 Group-Based Methods 273

14.2.7 Different Types of Food Recommender Systems 273

14.3 Implementation of Food Recommender System Using Content-Based Approach 276

14.3.1 Item Profile Representation 277

14.3.2 Information Retrieval 278

14.3.3 Word2vec 278

14.3.4 How are word2vec Embedding’s Obtained? 278

14.3.5 Obtaining word2vec Embeddings 279

14.3.6 Dataset 280

14.3.6.1 Data Preprocessing 280

14.3.7 Web Scrapping For Food List 280

14.3.7.1 Porter Stemming All Words 280

14.3.7.2 Filtering Our Ingredients 280

14.3.7.3 Final Data Frame with Dishes and Their Ingredients 281

14.3.7.4 Hamming Distance 281

14.3.7.5 Jaccard Distance 282

14.4 Results 282

14.5 Observations 283

14.6 Future Perspective of Recommender Systems 283

14.6.1 User Information Challenges 283

14.6.1.1 User Nutrition Information Uncertainty 283

14.6.1.2 User Rating Data Collection 284

14.6.2 Recommendation Algorithms Challenges 284

14.6.2.1 User Information Such as Likes/ Dislikes Food or Nutritional Needs 284

14.6.2.2 Recipe Databases 284

14.6.2.3 A Set of Constraints or Rules 285

14.6.3 Challenges Concerning Changing Eating Behavior of Consumers 285

14.6.4 Challenges Regarding Explanations and Visualizations 286

14.7 Conclusion 286

Acknowledgements 287

References 287

Part 4: Blockchain & IoT-Based Recommender Systems 291

15 A Trust-Based Recommender System Built on IoT Blockchain Network With Cognitive Framework 293
S. Porkodi and D. Kesavaraja

15.1 Introduction 294

15.1.1 Today and Tomorrow 294

15.1.2 Vision 294

15.1.3 Internet of Things 294

15.1.4 Blockchain 295

15.1.5 Cognitive Systems 296

15.1.6 Application 296

15.2 Technologies and its Combinations 297

15.2.1 IoT–Blockchain 297

15.2.2 IoT–Cognitive System 298

15.2.3 Blockchain–Cognitive System 298

15.2.4 IoT–Blockchain–Cognitive System 298

15.3 Crypto Currencies With IoT–Case Studies 299

15.4 Trust-Based Recommender System 299

15.4.1 Requirement 299

15.4.2 Things Management 302

15.4.3 Cognitive Process 303

15.5 Recommender System Platform 304

15.6 Conclusion and Future Directions 307

References 307

16 Development of a Recommender System HealthMudra Using Blockchain for Prevention of Diabetes 313
Rashmi Bhardwaj and Debabrata Datta

16.1 Introduction 314

16.2 Architecture of Blockchain 317

16.2.1 Definition of Blockchain 318

16.2.2 Structure of Blockchain 318

16.3 Role of HealthMudra in Diabetic 322

16.4 Blockchain Technology Solutions 324

16.4.1 Predictive Models of Health Data Analysis 325

16.5 Conclusions 325

References 326

Part 5: Healthcare Recommender Systems 329

17 Case Study 1: Health Care Recommender Systems 331
Usha Mittal, Nancy Singla and Geetika Gupta

17.1 Introduction 332

17.1.1 Health Care Recommender System 332

17.1.2 Parkinson’s Disease: Causes and Symptoms 333

17.1.3 Parkinson’s Disease: Treatment and Surgical Approaches 334

17.2 Review of Literature 335

17.2.1 Machine Learning Algorithms for Parkinson’s Data 337

17.2.2 Visualization 340

17.3 Recommender System for Parkinson’s Disease (PD) 341

17.3.1 How Will One Know When Parkinson’s has Progressed? 342

17.3.2 Dataset for Parkinson’s Disease (PD) 342

17.3.3 Feature Selection 343

17.3.4 Classification 343

17.3.4.1 Logistic Regression 343

17.3.4.2 K Nearest Neighbor (KNN) 343

17.3.4.3 Support Vector Machine (SVM) 344

17.3.4.4 Decision Tree 344

17.3.5 Train and Test Data 344

17.3.6 Recommender System 344

17.4 Future Perspectives 345

17.5 Conclusions 346

References 348

18 Temporal Change Analysis-Based Recommender System for Alzheimer Disease Classification 351
S. Naganandhini, P. Shanmugavadivu and M. Mary Shanthi Rani

18.1 Introduction 352

18.2 Related Work 352

18.3 Mechanism of TCA-RS-AD 353

18.4 Experimental Dataset 354

18.5 Neural Network 357

18.6 Conclusion 370

References 370

19 Regularization of Graphs: Sentiment Classification 373
R.S.M. Lakshmi Patibandla

19.1 Introduction 373

19.2 Neural Structured Learning 374

19.3 Some Neural Network Models 375

19.4 Experimental Results 377

19.4.1 Base Model 379

19.4.2 Graph Regularization 382

19.5 Conclusion 383

References 384

20 TSARS: A Tree-Similarity Algorithm-Based Agricultural Recommender System 387
Madhusree Kuanr, Puspanjali Mohapatra and Sasmita Subhadarsinee Choudhury

20.1 Introduction 388

20.2 Literature Survey 390

20.3 Research Gap 393

20.4 Problem Definitions 393

20.5 Methodology 393

20.6 Results & Discussion 394

20.6.1 Performance Evaluation 394

20.6.2 Time Complexity Analysis 396

20.7 Conclusion & Future Work 397

References 399

21 Influenceable Targets Recommendation Analyzing Social Activities in Egocentric Online Social Networks 401
Soumyadeep Debnath, Dhrubasish Sarkar and Dipankar Das

21.1 Introduction 402

21.2 Literature Review 403

21.3 Dataset Collection Process with Details 404

21.3.1 Main User’s Activities Data 405

21.3.2 Network Member’s Activities Data 405

21.3.3 Tools and Libraries for Data Collection 405

21.3.4 Details of the Datasets 406

21.4 Primary Preprocessing of Data 406

21.4.1 Language Detection and Translation 406

21.4.2 Tagged Tweeters Collection 407

21.4.3 Textual Noise Removal 407

21.4.4 Textual Spelling and Correction 407

21.5 Influence and Social Activities Analysis 407

21.5.1 Step 1: Targets Selection From OSMs 408

21.5.2 Step 3: Categories Classification of Social Contents 408

21.5.3 Step 4: Sentiments Analysis of Social Contents 408

21.6 Recommendation System 409

21.6.1 Secondary Preprocessing of Data 409

21.6.2 Recommendation Analyzing Contents of Social Activities 411

21.7 Top Most Influenceable Targets Evaluation 413

21.8 Conclusion 414

21.9 Future Scope 415

References 415

Index 417