Media Production, Delivery and Interaction for Platform Independent Systems (eBook)

Oliver Schreer, Fraunhofer Heinrich-Hertz-Institut, Berlin, Germany Dr. Schreer is Scientific Project Manager of the "Immersive Media & 3D Video" Group in the Image Processing Department at HHI, Berlin. He is responsible for the lead of European Research projects (e.g. FascinatE project) and coordinating research on 3D video processing and immersive media. Graham Thomas, BBC Research & Development, London, UK Dr. Thomas is Principal Research Engineer and Section Lead of the Production Magic department at BBC R&D. He leads a team of engineers developing 3D image processing and graphics techniques for TV production. His work has led to many commercial products, e.g. the Piero sports graphics system, which won a Queen's Award in 2011. He holds 20 patents and is Visiting Professor at University of Surrey. Ben Shirley, University of Salford, UK Ben Shirley is Senior Lecturer at the University of Salford and Academic Lead for the Digital Innovation Cluster in the University's College of Science & Technology at MediaCityUK. He has been principal investigator for a range of broadcast related research projects including leading Salford's work in the EU FascinatE project. He teaches audio technology and broadcast engineering at undergraduate and postgraduate level. Georg Thallinger, Joanneum Research, Graz, Austria Dr. Thallinger is Head of the research group "Audiovisual Media" at JOANNEUM RESEARCH, Austria. He oversees a group active in the field of digital media with a focus on video and image processing as well as metadata for applications in media production and archiving, film restoration, media monitoring and security. Dr. Thallinger is involved in the EU FascinatE project. Jean-Francois Macq, Alcatel-Lucent, Antwerp, Belgium Dr. Macq is a Senior Research Engineer at Bell Labs, the research organization of Alcatel-Lucent. He has been responsible for several national and international research projects in the field of video coding and delivery, currently including the EU FascinatE project. Javier Ruiz-Hidalgo, Universitat Politècnica de Catalunya (UPC), Barcelona, Spain Dr. Ruiz-Hidalgo is an Associate Professor at the Universitat Politècnica de Catalunya. He has been involved in various European Projects as a principal researcher from the Image Processing Group at UPC, including the EU FascinatE project. Omar Niamut, Netherlands Organization for Applied Scientific Research (TNO), Delft, The Netherlands Dr. Niamut is a Senior Research Scientist in the "Media Networks and Services" Group at TNO. He has advised the European Parliament on the harmonization of mobile TV and was editor of the technology report on mobile TV in the European Commission-assigned European Mobile Broadcast Council. He participated in ETSI TISPAN standardization with over 200 contributions on interactive services for next generation IPTV and advised the Singapore government on the use of IPTV standards. He has been involved in a number of European projects, including the Fascinate project. Dr. Niamut holds 15 patents.

List of Editors and Contributors xiii

List of Abbreviations xvii

Notations xxiii

1 Introduction 1
Oliver Schreer, Jean-Francois Macq, Omar Aziz Niamut, Javier Ruiz-Hidalgo, Ben Shirley, Georg Thallinger and Graham Thomas

2 State-of-the-Art and Challenges in Media Production, Broadcast and Delivery 5
Graham Thomas, Arvid Engstrom, Jean-Francois Macq, Omar Aziz Niamut, Ben Shirley and Richard Salmon

2.1 Introduction 5

2.2 Video Fundamentals and Acquisition Technology 7

2.3 Audio Fundamentals and Acquisition Technology 21

2.4 Live Programme Production 34

2.5 Coding and Delivery 43

2.6 Display Technology 50

2.7 Audio Reproduction Technology 56

2.8 Use of Archive Material 62

2.9.1 Limitations of Current Production and Delivery Approaches 65

2.10 Conclusion 68

3 Video Acquisition 74
Oliver Schreer, Ingo Feldmann, Richard Salmon, Johannes Steurer and Graham Thomas

3.1 Introduction 74

3.2 Ultra-High Definition Panoramic Video Acquisition 75

3.3 Use of Conventional Video Content to Enhance Panoramic Video 94

3.4 High Frame Rate Video 102

3.5 High Dynamic Range Video 112

3.6 Conclusion 125

4 Platform Independent Audio 130
Ben Shirley, Rob Oldfield, Frank Melchior and Johann-Markus Batke

4.1 Introduction 130

4.2 Terms and Definitions 132

4.3 Definition of the Problem Space 135

4.4 Scene Representation 144

4.5 Scene Acquisition 149

4.6 Scene Reproduction 153

4.7 Existing Systems 156

4.8 Conclusion 161

5 Semi-Automatic Content Annotation 166
Werner Bailer, Marco Masetti, Goranka Zoric, Marcus Thaler and Georg Thallinger

5.1 Introduction 166

5.2 Metadata Models and Analysis Architectures 170

5.3 Domain-independent Saliency 177

5.4 Person Detection and Tracking 180

5.5 Online Detection of Concepts and Actions 189

5.6 Supporting Annotation for Automated Production 195

5.7 Conclusion 204

6 Virtual Director 209
Rene Kaiser and Wolfgang Weiss

6.1 Introduction 209

6.2 Implementation Approaches 219

6.3 Example Architecture and Workflow 225

6.4 Virtual Director Subprocesses 230

6.5 Behaviour Engineering: Production Grammar 237

6.6 Virtual Director: Example Prototype 243

6.7 Conclusion 251

7 Scalable Delivery of Navigable and Ultra-High Resolution Video 260
Jean-Francois Macq, Patrice Rondao Alface, Ray van Brandenburg, Omar Aziz Niamut, Martin Prins and Nico Verzijp

7.1 Introduction 260

7.2 Delivery of Format-Agnostic Content: Key Concepts and State-of-the-Art 262

7.3 Spatial Random Access in Video Coding 267

7.4 Models for Adaptive Tile-based Representation and Delivery 276

7.5 Segment-based Adaptive Transport 281

7.6 Conclusion 294

8 Interactive Rendering 298
Javier Ruiz-Hidalgo, Malte Borsum, Axel Kochale and Goranka Zoric

8.1 Introduction 298

8.2 Format-Agnostic Rendering 299

8.3 Device-less Interaction for Rendering Control 311

8.4 Conclusions 331

9 Application Scenarios and Deployment Domains 337
Omar Aziz Niamut, Arvid Engstrom, Axel Kochale, Jean-Francois Macq, Graham Thomas and Goranka Zoric

9.1 Introduction 337

9.2 Application Scenarios 338

9.3 Deployment in the Production Domain 340

9.4 Deployment in the Network Domain 347

9.5 Deployment in the Device Domain 351

9.5.1 Device Capabilities 351

9.6 Deployment in the User Domain 356

9.7 Conclusion 357

References 357

Index 359