Data Engineering and the Death of Visual ETL

Session Number: 8812
Track: Big Data & Data Warehousing
Sub-Categorization: Data Engineering
Session Type: Solutions
Primary Presenter: Stewart Bryson [Owner and Co-Founder - Red Pill Analytics]
Time: Jun 24, 2019 (11:45 AM - 12:45 PM)
Room: 310, Level 3

Speaker Bio: Stewart Bryson is the owner and co-founder of Red Pill Analytics, and he has been designing and implementing BI systems since 1996. He is an Oracle ACE Director, an internationally recognized writer and speaker, and co-host of the video and audio podcast See Data Differently.

Stewart founded Transcendent Data in 2004, building a consulting practice around his considerable knowledge of best practices and Oracle BI and DW products. In 2008, he joined Rittman Mead Consulting as the US CEO and later as the global chief innovation officer. In 2014, along with Kevin McGinley, Stewart founded Red Pill Analytics with the desire to help customers see data differently.
Technologies or Products Used: Interactive Notebooks, Streaming Framework

Session Summary for Attendees:  A funny thing happened on the way to modern, data-driven analytics: we ditched the GUI for ETL development. There are drawbacks to moving data using code instead of clicks, especially for organizations heavily invested in visual tools. But writing code has an upside as well.

In this presentation, we’ll explore the data engineering discipline and how it compares to conventional ETL. We’ll look at classic batch as well as modern streaming use cases, and we’ll consider DataOps requirements around testing and delivery where code really shines. We’ll do it all inside of an interactive notebook.