PM1: Big Data Engineering

This tutorial presents Big Data Engineering (BDE), which is defined as the practical application of a systematic, disciplined, quantifiable approach to the analysis, design, construction, operation, and maintenance of Big Data solutions.

BDE is a holistic method focusing on 8 crucial areas: Methodology, Program, Governance, Resources, Quality, Risk Mitigation, KPI & Financials, and Practice. BDE also systematically addresses the lifecycle of Big Data solutioning in 12 stages: Plan, Requirement, Analysis, Modeling, Platform, Design, Development, Integration, Testing, Runtime, Deployment, and Operation.

Each of these 12 stages comprises individual elements as subdisciplines. For example, the NoSQL platform options include key-value, column-based, document-oriented, graph, NewSQL, and In-memory stores. Case studies and working examples will be discussed in great details in the session to illustrate the pragmatic use of BDE in the real-world implementations. Best practices and lessons learned are articulated as well.

Topics you will learn about include:

Engineering discipline
Lifecycle
Methodology
Governance
Best practices

Tony Shan is a renowned visionary and thought leader incubating and nurturing interdisciplinary practice and enablement on emerging technologies like IoT, big data, cloud, and bots. He drives large-scale transformations of the most complex information systems at Fortune 500 companies. He leads pragmatic innovation for enterprise digitization in consulting.

He serves as an editor and on the advisory board of journals, as the chair of events, and as a judge in IT competitions. He speaks and organizes in various conferences and has authored hundreds of publications. As an intrapreneur/entrepreneur, he also founded several startups and user groups/forums.