Handbook on Networked Multipoint Multimedia Conferencing and Multistream Immersive Telepresence using SIP

Radhika Ranjan Roy has been an electronics engineer, United States Army Research, United States Command, Control, Communications, Computers, Combat Systems, Intelligence, Surveillance, and Reconnaissance (C5ISR)-S&TCD Laboratories (previously known as CERDEC), Aberdeen Proving Ground (APG), Maryland, United States since 2009. Dr. Roy is leading research and development efforts in the development of scalable large-scale SIP-based VoIP/Multimedia networks and services, cybersecurity, artificial intelligence and machine/deep learning (AI/ML/DL) architecture, mobile ad hoc networks (MANETs), Peer-to-Peer (P2P) networks, cybersecurity detecting application software and network vulnerability, jamming detection, supporting an array of the Army/ Department of Defense’s Nationwide and Worldwide Warfighter Networking Architectures and participating in technical standards development in Multimedia/Real-Time Services Collaboration, IPv6, Radio Communications, Enterprise Services Management, and Information Transfer of Department of Defense (DoD) Technical Working Groups (TWGs). He received his PhD in Electrical Engineering with Major in Computer Communications from the City University of New York (CUNY), NY, United States in 1984 and his MS in Electrical Engineering from the Northeastern University, Boston, MA, United States in 1978. He received his BS in Electrical Engineering from the Bangladesh University of Engineering & Technology (BUET), Dhaka, Bangladesh in 1967. He was born in the renowned country town of Derai, Bangladesh. Prior to joining CERDEC, Dr. Roy worked as the lead system engineer at CACI, Eatontown, NJ from 2007 to 2009 and developed Army Technical Resource Model (TRM), Army Enterprise Architecture (AEA), DoD Architecture Framework (DoDAF), and Army LandWarNet (LWN) Capability Sets, and technical standards for Joint Tactical Radio System (JTRS), Mobile IPv6, MANET, and SIP, supporting Army Chief Information Officer (CIO)/G-6. Dr. Roy worked as senior system engineer, SAIC, Abingdon, MD from 2004 to 2007, supporting Modeling, Simulations, Architectures, and System Engineering of many Army projects: WIN-T, FCS, and JNN. During his career, Dr. Roy worked in AT&T/Bell Laboratories, Middletown, NJ as senior consultant from 1990 to 2004 and led a team of engineers in designing AT&T’s Worldwide SIP-based VoIP/Multimedia Communications Network Architecture, consisting of wired and wireless, from the preparation of Request for Information (RFI) to the evaluation of vendor RFI responses and interactions with all selected major vendors related to their products. He participated in and contributed to the development of VoIP/H.323/SIP multimedia standards in ITU-T, IETF, ATM, and Frame Relay standard organizations. Dr. Roy worked as senior principal engineer in CSC, Falls Church, VA from 1984 to 1990 and worked in the design and performance analysis of the US Treasury nationwide X.25 packet-switching network. In addition, he designed many network architectures of many proposed U.S. Government and Commercial Worldwide and Nationwide Networks: Department of State Telecommunications Network (DOSTN), U.S. Secret Service Satellite Network, Veteran Communications Network, and Ford Company’s Dealership Network. Prior to CSC, he worked from 1967 to 1977 as deputy director, Design, in PDP, Dhaka, Bangladesh. Dr. Roy’s research interests include the areas of artificial intelligence, machine/deep learning, blockchain cloud and fog computing, mobile ad hoc networks, multimedia communications, peer-topeer networking, and quality-of-service. He has published over 60 technical papers and is holding or pending over 35 patents. He also participates in many IETF working groups. Dr. Roy authored three books Handbook on SDP for Multimedia Session Negotiations: SIP and WebRTC Telephony (CRC Press/Taylor & Francis, 2018), Handbook on Session Initiation Protocol: Networked Multimedia Communications for IP Telephony (CRC Press/Taylor & Francis, 2016), and Handbook of Mobile Ad Hoc Networks for Mobility Models, Springer, 2010. He lives in the historical district of Howell Township, New Jersy, with his wife Jharna.

Contents

Preface...........................................................................................................................................xxxi

Author...........................................................................................................................................xxxv

Chapter 1 Scalable Networked Multipoint Multimedia Conferencing and Telepresence..............1

1.1 Networked Multipoint Multimedia Conferencing..............................................1

1.2 Basic Conferencing Protocol Architecture.........................................................4

1.3 Third-Party Multiparty Conferencing.............................................................. 12

1.4 Multistream Immersive Telepresence...............................................................23

1.5 Scalable Centralized Multipoint Multimedia Conferencing Systems

and Protocols Framework.................................................................................24

1.6 Terminology.....................................................................................................28

1.7 Summary..........................................................................................................28

1.8 Problems........................................................................................................... 41

References................................................................................................................... 42

Chapter 2 Centralized Conferencing System Architecture......................................................... 43

2.1 Introduction...................................................................................................... 43

2.2 Convention........................................................................................................ 43

2.3 Terminology..................................................................................................... 43

2.4 Overview..........................................................................................................44

2.5 Centralized Conferencing Data........................................................................ 45

2.6 Centralized Conferencing Constructs and Identifiers...................................... 47

2.7 Conferencing System Realization....................................................................50

2.8 Conferencing Mechanisms............................................................................... 55

2.9 Conference Scenario Realizations....................................................................56

2.10 Relationships between SIP and Centralized Conferencing System

Frameworks......................................................................................................69

2.11 Security Considerations....................................................................................69

2.12 Summary.......................................................................................................... 71

2.13 Problems........................................................................................................... 72

Chapter 3 Media Server Control Architecture............................................................................. 73

3.1 Introduction...................................................................................................... 73

3.2 Terminology..................................................................................................... 75

3.3 Architecture Overview..................................................................................... 75

3.4 NSIP Usage.......................................................................................................77

3.5 Media Control for IVR Services...................................................................... 78

3.6 Media Control for Conferencing Services........................................................80

3.7 Security Considerations....................................................................................85

3.8 Summary..........................................................................................................86

3.9 Problems...........................................................................................................87

Chapter 4 Conferencing Information Data Model for Centralized Conferencing.......................89

4.1 Introduction......................................................................................................89

4.2 Terminology.....................................................................................................89

4.3 Overview..........................................................................................................89

Comparison.......................................................................... 91

4.4 Data Model Definition......................................................................................92

4.5 RELAX NG Schema...................................................................................... 103

4.6 XML Schema Extensibility............................................................................ 103

4.7 XML Example................................................................................................ 103

4.8 Non-Normative RELAX NG Schema in XML Syntax (Appendix A of

RFC 6501)....................................................................................................... 103

4.9 Relax Schema in XML Syntax (Appendix B of RFC 6501).......................... 103

4.10 Security Considerations.................................................................................. 104

4.11 Summary........................................................................................................ 108

4.12 Problems......................................................................................................... 109

References................................................................................................................. 111

Chapter 5 Centralized Conferencing Manipulation Protocol.................................................... 129

5.1 Introduction.................................................................................................... 129

5.2 Conventions and Terminology........................................................................ 129

5.3 XCON Conference Control System Architecture.......................................... 129

5.4 Protocol Overview.......................................................................................... 131

5.5 CCMP Messages............................................................................................ 134

5.6 Example of CCMP in Action......................................................................... 153

5.7 Locating a Conference Server........................................................................ 162

5.8 Managing Notifications.................................................................................. 163

5.9 HTTP Transport............................................................................................. 165

5.10 Security Considerations.................................................................................. 166

5.11 XML Schema................................................................................................. 168

5.12 IANA Considerations..................................................................................... 169

5.13 Summary........................................................................................................ 176

5.14 Evaluation of Other Protocol Models and Transports Considered for

CCMP (Appendix A of RFC 6503)................................................................ 176

5.15 Problems......................................................................................................... 178

References................................................................................................................. 179

Chapter 6 Binary Floor Control Protocol.................................................................................. 181

6.1 Introduction.................................................................................................... 181

6.2 Terminology................................................................................................... 181

6.3 Scope.............................................................................................................. 181

6.4 Overview of Operation................................................................................... 183

6.5 Packet Format................................................................................................. 186

6.6 Transport........................................................................................................200

6.7 Lower-Layer Security..................................................................................... 201

6.8 Protocol Transactions..................................................................................... 201

6.9 Authentication and Authorization..................................................................202

6.10 Floor Participant Operations..........................................................................202

6.11 Chair Operations............................................................................................205

6.12.1 Requesting Information about Floors...............................................206

6.13 Floor Control Server Operations....................................................................208

6.14 Security Considerations.................................................................................. 214

6.15 BFCP Connection Establishment................................................................... 215

6.16 IANA Considerations..................................................................................... 218

6.17 Summary........................................................................................................ 221

6.18 Problems.........................................................................................................222

Chapter 7 XCON Notification Service.......................................................................................225

7.1 Introduction....................................................................................................225

7.2 Terminology...................................................................................................225

7.3 Notification Formats.......................................................................................225

7.4 Full Notifications............................................................................................225

7.5 Partial Notifications........................................................................................227

7.6 IANA Considerations.....................................................................................229

7.7 Security Considerations..................................................................................229

7.7.1 Connection Security..........................................................................230

7.7.2 Authorization Considerations ...........................................................230

7.8 Summary........................................................................................................230

7.9 Problems......................................................................................................... 231

Reference................................................................................................................... 231

Chapter 8 Media Channel Control Framework.......................................................................... 233

8.1 Introduction.................................................................................................... 233

8.2 CFW Capabilities........................................................................................... 233

8.3 Overview........................................................................................................ 235

8.4 Control Channel Setup................................................................................... 238

8.4.1 Control Client SIP UAC Behavior..................................................... 238

8.4.2 Control Server SIP UAS Behavior.................................................... 239

8.5 Establishing Media Streams – Control Client SIP UAC Behavior.................240

8.6 Control Framework Interactions..................................................................... 241

8.7 Response Code Descriptions.......................................................................... 247

8.8 Control Packages............................................................................................ 247

8.9 Formal Syntax................................................................................................249

8.10 Examples........................................................................................................ 251

8.11 Extensibility....................................................................................................254

8.12 Security Considerations..................................................................................254

8.12.1 Session Establishment.......................................................................254

8.13 IANA Considerations.....................................................................................256

8.14 Common Package Components (Appendix A of RFC 6230).........................256

8.15 Summary........................................................................................................ 257

8.16 Problems......................................................................................................... 257

Chapter 9 Mixer Package for Media Control Channel Framework........................................... 259

9.1 Introduction.................................................................................................... 259

9.2 Conventions and Terminology........................................................................260

9.3 Control Package Definition............................................................................260

9.4 Element Definitions........................................................................................ 262

9.5 Formal Syntax................................................................................................290

9.6 Examples........................................................................................................ 291

9.6.1 AS-MS Framework Interaction Examples........................................ 291

9.6.1.1 Creating a Conference Mixer and Joining a

Participant..........................................................................309

9.6.1.2 Receiving Active Talker Notifications...............................309

9.6.1.3 Conference Termination....................................................309

9.6.2 Mixing Examples.............................................................................. 310

9.6.2.1 Audio Conferencing........................................................... 310

9.6.2.2 Bridging Connections........................................................ 312

xvi Contents

9.6.2.3 Video Conferencing........................................................... 312

9.7 Security Considerations.................................................................................. 313

9.8 IANA Considerations..................................................................................... 315

9.9 Summary........................................................................................................ 315

9.10 Problems......................................................................................................... 315

References................................................................................................................. 316

Chapter 10 Media Session Recording.......................................................................................... 317

10.1 Use Cases and Requirements for SIP-Based Media Recording................... 317

10.2 Media Recording Architecture....................................................................324

10.3 Media Recording Metadata.......................................................................... 331

XML Body.................................................................344

10.3.9 XML Schema Definition for Recording Metadata....................... 345

10.3.10 Security Considerations................................................................348

10.3.11 IANA Considerations...................................................................349

10.4 Media Session Recoding Protocol...............................................................349

10.5 Summary...................................................................................................... 371

10.6 Problems....................................................................................................... 372

References................................................................................................................. 373

Chapter 11 Media Resource Brokering....................................................................................... 375

11.1 Introduction.................................................................................................. 375

11.2 Conventions and Terminology..................................................................... 375

11.3 Problem Discussion...................................................................................... 377

11.4 Deployment Scenario Options..................................................................... 378

11.5 MRB Interface Definitions.......................................................................... 381

11.6 MRB Acting as a B2BUA............................................................................405

11.7 Multimodal MRB Implementations.............................................................405

11.8 Relative Merits of Query Mode, IAMM, and IUMM.................................406

11.9 Examples......................................................................................................407

11.10 Media Service Resource Publisher Interface XML Schema....................... 425

11.11 Media Service Resource Consumer Interface XML Schema......................444

11.12 Security Considerations............................................................................... 461

11.13 IANA Considerations...................................................................................463

11.14 Summary......................................................................................................463

11.15 Problems.......................................................................................................464

References.................................................................................................................465

Chapter 12 Media Control Channel Framework for Interactive Voice Response ....................... 467

12.1 Introduction.................................................................................................. 467

12.2 Conventions and Terminology.....................................................................469

12.3 Control Package Definition..........................................................................469

12.4 Element Definitions...................................................................................... 471

12.5 Formal Syntax.............................................................................................. 511

12.6 Examples...................................................................................................... 534

12.8 IANA Considerations...................................................................................544

12.9 Using VoiceXML as a Dialog Language.....................................................544

12.10 Summary...................................................................................................... 549

12.11 Problems....................................................................................................... 550

Chapter 13 SIP Interface to VoiceXML Media Services............................................................ 551

13.1 Introduction.................................................................................................. 551

13.2 VoiceXML Session Establishment and Termination................................... 553

13.3 Media Support..............................................................................................560

13.4 Returning Data to the Application Server................................................... 563

13.5 Outbound Calling.........................................................................................565

13.6 Call Transfer................................................................................................. 565

13.7 Security Considerations............................................................................... 569

13.8 IANA Considerations...................................................................................569

13.9 Summary......................................................................................................569

13.10 Problems....................................................................................................... 570

References................................................................................................................. 571

Chapter 14 Media Resource Control Protocol Version 2............................................................. 573

14.1 Introduction.................................................................................................. 573

14.2 Conventions.................................................................................................. 574

14.3 Architecture................................................................................................. 575

14.4 MRCPv2 Basics........................................................................................... 576

14.5 MRCPv2 Specification................................................................................. 583

14.5.5 Events........................................................................................... 587

14.6 MRCPv2 Generic Methods, Headers, and Result Structure........................ 588

14.7 Resource Discovery..................................................................................... 597

14.8 Speech Synthesizer Resource...................................................................... 597

14.10 Recorder Resource....................................................................................... 656

14.11 Speaker Verification and Identification........................................................663

14.12 Security Considerations...............................................................................684

14.13 IANA Considerations...................................................................................686

14.14 Examples......................................................................................................686

14.15 ABNF Normative Definition........................................................................697

14.16 XML Schemas.............................................................................................706

14.17 Summary...................................................................................................... 711

14.18 Problems....................................................................................................... 713

References................................................................................................................. 714

Chapter 15 Media Control Channel Framework (CFW) Call Flow Examples........................... 715

15.1 Introduction.................................................................................................. 715

15.2 Conventions.................................................................................................. 715

15.3 Terminology................................................................................................. 716

15.4 A Practical Approach................................................................................... 716

15.4.1 State Diagrams............................................................................. 716

15.5 Control Channel Establishment................................................................... 718

15.6 Use-Case Scenarios and Examples.............................................................. 726

15.7 Media Resource Brokering..........................................................................792

15.8 Security Considerations...............................................................................820

15.9 Summary......................................................................................................827

15.10 Problems.......................................................................................................830

Reference................................................................................................................... 831

Chapter 16 Multistream Immersive Telepresence Conferencing Systems.................................. 833

16.1 Requirements for Telepresence Multistreams.............................................. 833

16.2 Use Cases for Telepresence Multistreams.................................................... 837

16.3 Framework for Telepresence Multistreams.................................................. 845

16.4 XML Schema for the CLUE Data Model....................................................885

16.5 CLUE Protocol............................................................................................. 933

16.6 CLUE Protocol Data Channel.....................................................................966

16.7 Mapping RTP Streams to CLUE Media Captures.......................................980

16.8 Session Signaling for Controlling Multiple Streams for Telepresence

(CLUE)......................................................................................................... 983

16.9 Summary......................................................................................................997

16.10 Problems..................................................................................................... 1001

References...............................................................................................................1002

Appendix....................................................................................................................................... 1005

Index............................................................................................................................................. 1013