97 Things Every Data Engineer Should Know

Download or read book 97 Things Every Data Engineer Should Know written by Tobias Macey and published by "O'Reilly Media, Inc.". This book was released on 2021-06-11 with total page 263 pages. Available in PDF, EPUB and Kindle. Book excerpt: Take advantage of the sky-high demand for data engineers today. With this in-depth book, current and aspiring engineers will learn powerful, real-world best practices for managing data big and small. Contributors from Google, Microsoft, IBM, Facebook, Databricks, and GitHub share their experiences and lessons learned for overcoming a variety of specific and often nagging challenges. Edited by Tobias Macey from MIT Open Learning, this book presents 97 concise and useful tips for cleaning, prepping, wrangling, storing, processing, and ingesting data. Data engineers, data architects, data team managers, data scientists, machine learning engineers, and software engineers will greatly benefit from the wisdom and experience of their peers. Projects include: Building pipelines Stream processing Data privacy and security Data governance and lineage Data storage and architecture Ecosystem of modern tools Data team makeup and culture Career advice.